28903

Linux: OS support for an unprivileged interprocess bus

Question:

I'm trying to find a simple solution for high-performance broadcast/multicast communication between unprivileged processes running on a Linux host. I'm looking for a solution that is 1) simple, 2) unprivileged (no root), 3) language-independent, 4) packet-oriented and 5) efficient (Gbit/s and up).

To put this in context, my existing code simply uses UDP sockets for unicast communication, which neatly matches the above requirements (except being unicast). I've looked into expanding this to multicast by having multiple programs listen to the same UDP port (using SO_REUSEADDR and/or SO_REUSEPORT), but this doesn't actually distribute copies of the packet to all the processes.

I've also looked into using loopback broadcast (127.255.255.255) to reach multiple listening processes, but it seems that I'll need to bind to multiple IP-addresses on the loopback device for this to work, and adding these addresses requires root.

Answer1:

Expanding on Pete's suggestion, I've found the following solution, which is not <em>too</em> complex.

The following Python 2 program implements a simple chat program over multicast/loopback. Tested on Linux 3.13 / Ubuntu 14.04.

<pre class="lang-python prettyprint-override">import os, socket, sys # Use an administratively scoped multicast IP (RFC 2365). mcast_group = '239.0.0.0' port = 1234 # Communicate over the loopback interface. ifc = '127.0.0.1' def send(data): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0) # Send over loopback interface. sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, socket.inet_aton(ifc)) sock.sendto(data, (mcast_group, port)) def listen(): sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0) # Join group. sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, socket.inet_aton(mcast_group) + socket.inet_aton(ifc)) # Allow multiple subscribers. sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) sock.bind((mcast_group, port)) while True: data, remote = sock.recvfrom(1024) print '#%d (%s:%d): %s' % ( (os.getpid(),) + remote + (data,) ) args = ' '.join(sys.argv[1:]) if args: send(args) else: listen()

This expands trivially to communication across the local broadcast segment (LAN) by setting ifc = '0.0.0.0' and unsetting the IP_MULTICAST_IF option.

<strong>Receiving one's own traffic</strong>

The only problem is that there appears to be no simple way to join a group and send to everyone <em>else</em> in the group without getting your own traffic as well (even by using the same socket for listening and sending).

The IPPROTO_IP socket option IP_MULTICAST_LOOP does not work; it has no effect when using the loopback interface, and when communicating over the network it prevents other local clients from receiving the messages.

I can work around this by filtering on the source address, though.

(As a side note, I just realized that QEMU includes <a href="https://people.gnome.org/~markmc/qemu-networking.html" rel="nofollow">native support</a> for sending virtual network traffic using multicast. <a href="http://comments.gmane.org/gmane.comp.emulators.qemu/198572" rel="nofollow">Faced with the above problem,</a> the QEMU developers also discussed filtering based on source address, though it appears that they never actually did anything about it. As a result, the QEMU VM receives a copy of its own outgoing traffic, though the traffic is usually rejected in the network stack.)

Recommend

  • Create HTML form from JsonData and make it editable
  • How to update nested objects in Elasticsearch 2.2 script via Java API
  • Displaying a server side rendered reactJS component within a PHP application
  • mysql: select query when the column that has condition on has NULL value
  • Adding dependencies to a custom gradle plugin
  • (XCode 4.0.2) Archive build (build for distribution on app-store) armv6 warning
  • iOS client server approach
  • Question regarding the ExtJS License [closed]
  • Add new accepted protocol to URL/link validation in SharePoint 2010
  • Android SearchView requires two clicks to expand the view
  • chrome devtools inconsistency array length
  • sending email using “bcc” without “to” in java application
  • How to restrict number of concurrent processes?
  • DotNetOpenAuth - how to uniquely identify Google users?
  • Google OAuth2 for an web application hosted behind NAT (intranet server without public IP)
  • Is an if-let or a normal if condition better?
  • How to resolve docker host names (/etc/hosts) in containers
  • Should I be afraid to use UDP to make a client/server broadcast talk?
  • Is there a way to choose which files are displayed to the user via the standard OPENFILE dialogs?
  • python: forcing relative imports to search from script file
  • Knockout custom binding handler
  • Eclipse MTJ doesn't see Java ME SDK 3.0 devices
  • blade.php method outputting it's result to the form
  • How do I include a SWC in an AS2 Flash project?
  • How to add a focus style to an editable ComboBox in WPF
  • Firefox Extension - Monitor refresh and change of tab
  • How do I superscript characters in a UIButton?
  • Cannot resolve symbol 'MyApi'
  • How to get address from latitude and longitude android google map v2 [duplicate]
  • Is my CUDA kernel really runs on device or is being mistekenly executed by host in emulation?
  • jQuery show() function is not executed in Safari if submit handler returns true
  • Sony Xperia Z Tablet not found by adb
  • Can I make an Android app that runs a web view in Chrome 39?
  • R: gsub and capture
  • jqPlot EnhancedLegendRenderer plugin does not toggle series for Pie charts
  • Comma separated Values
  • How to get Windows thread pool to call class member function?
  • Is there any way to bind data to data.frame by some index?
  • Does armcc optimizes non-volatile variables with -O0?
  • How to load view controller without button in storyboard?