Python

Start the two in either order, in two terminals: stream_connect waits up to its timeout for a server to create the stream and accept. stream_create returns a Listener, and the connections are Conns; both are context managers. The pair makes the exchange every example on this site makes, the message ping, answered in upper case: this client works with any language's server, and this server with any language's client (every pair of languages is tested).

import fipc

# rings of 1 MiB each way
with fipc.stream_create("my_channel", 1 << 20) as listener:
    with listener.accept() as conn:  # waits for a client
        message = conn.recv()
        conn.send(message.upper())

import fipc

with fipc.stream_connect("my_channel", timeout_ms=5000) as conn:
    conn.send(b"ping")
    print(conn.recv())  # b'PING'

Worker processes speed a job up only if talking to them is cheap. Every task crosses between processes twice, and that communication overhead, not the work, sets the limit. With one dispatcher handing out tasks of time t to P workers:

$$S=\frac{1}{\frac{c}{t}+\frac{1+R/t}{P}}\le \frac{t}{c}$$

S

speedup: how many times faster the job finishes than in one process

P

worker processes

t

time one task takes to compute

R

round trip: send a task to a worker and get its result back

c

the dispatcher's own time per task, handing it out and taking its result

However many workers, the job runs at most t / c times as fast. Python's own channels cost 20 to 70 µs a round trip and 8 to 56 µs of the dispatcher's time a task: tasks of 20 µs gain nothing from a pool fed through multiprocessing.Queue. FastIPC's round trip, Python to Python, is about 2 µs, so tasks of tens of microseconds scale across the cores. Every point on the chart is measured.

A connection carries messages: send takes bytes of any size, and recv returns each message whole. This server echoes every message back until its client leaves, which ends its next recv with DISCONNECTED.

import fipc

# rings of 1 MiB each way: a power of two, 1 KiB to 2 GiB
with fipc.stream_create("demo", 1 << 20) as listener:
    with listener.accept() as conn:
        try:
            while True:
                msg = conn.recv()     # bytes, any size
                conn.send(msg)        # echo it back
        except fipc.FipcError as e:
            if e.result != fipc.Result.DISCONNECTED:
                raise
            print("client gone")

import fipc

with fipc.stream_connect("demo", timeout_ms=5000) as conn:
    for word in (b"hello", b"shared", b"memory"):
        conn.send(word)
        print(conn.recv())

Write into a memoryview of the ring and commit; read a read-only memoryview of the ring and release it. A view stays valid until its release, the next receive, or close, and keeps its Conn referenced: the garbage collector never closes a connection under a view.

buf = conn.send_acquire(5)    # writable memoryview
buf[:] = b"hello"
conn.send_commit(5)

view = conn.recv_acquire()    # read-only memoryview
handle(view)
conn.recv_release()           # frees its room

A broadcast sends every message of one publisher to each of its subscribers, in any processes of the user; each message is stored once, in memory every subscriber maps. Messages are of one piece, up to a quarter of the capacity. broadcast_create returns the publisher, a Conn and the broadcast's one handle, which the with block closes; its close ends the broadcast and frees the name. The publisher's process admits each subscriber by itself, on a thread of the library's: the publisher only waits for the first one (publisher.wait_for_members(1)) before it sends tick 1 to tick 3. The subscriber joins gated (broadcast_join with SUBSCRIBER_GATED), so it misses nothing (the publisher waits for it), and receives until DISCONNECTED, which comes after the last message. A lossy subscriber (SUBSCRIBER_LOSSY) never holds the publisher back: one that falls a lap behind gets OVERRUN, and the error's lost says how many messages it lost. A gated one that holds the publisher back for the broadcast's max_stall_ms is unsubscribed (UNSUBSCRIBED).

import fipc

# 64 KiB of messages, up to 8 subscribers; one that holds the publisher back 5 s is dropped
with fipc.broadcast_create("my_broadcast", 1 << 16, 8, 5000) as publisher:
    publisher.wait_for_members(1)  # waits for the first subscriber
    for i in (1, 2, 3):
        publisher.send(f"tick {i}".encode())
# closing the publisher ends the broadcast: subscribers get DISCONNECTED after the ticks

import fipc

# gated: misses nothing (the publisher waits for it)
with fipc.broadcast_join("my_broadcast", fipc.SUBSCRIBER_GATED, timeout_ms=5000) as subscriber:
    try:
        while True:
            print(subscriber.recv().decode())  # tick 1, tick 2, tick 3
    except fipc.FipcError as e:
        if e.result != fipc.Result.DISCONNECTED:  # the publisher closed
            raise

A work queue takes messages from up to 63 producers and hands each to one of up to 64 consumers, in any processes of the user. Each producer has a ring of its own; a consumer takes a message of any producer, and the queue promises no order. Messages are of one piece, up to a quarter of the capacity, and wait in the queue until a consumer takes them. Each message goes to one consumer, and again, to another, only if that consumer's process ends before acknowledging it (delivery, below). Here the producer creates the queue (queue_create: a Queue), joins it as its producer (queue_join with PRODUCER: the owner's process admits every joiner by itself, its own included), sends job 1 to job 3, which wait in the queue, and waits for the consumer (queue.wait_for_members(CONSUMER, 1)). The queue's close ends it: the consumer receives what is left, then DISCONNECTED, and a producer's sends raise DISCONNECTED from then on; a producer's or a consumer's close never ends the queue. A Queue never closes itself (it has no finalizer): a queue lives until close() or the end of its process, and dropping the object doesn't end it.

import fipc

# 4 KiB of messages per producer, 1 producer and 1 consumer: this process owns the queue and is its producer
with fipc.queue_create("my_queue", 4096, 1, 1) as queue:
    with fipc.queue_join("my_queue", fipc.PRODUCER) as producer:
        for i in (1, 2, 3):
            producer.send(f"job {i}".encode())  # waits in the queue until a consumer takes it
        queue.wait_for_members(fipc.CONSUMER, 1)  # waits for the consumer to join
# closing the queue ends it: the consumer gets DISCONNECTED after the jobs

import fipc

with fipc.queue_join("my_queue", fipc.CONSUMER, timeout_ms=5000) as consumer:
    try:
        while True:
            print(consumer.recv().decode())  # job 1, job 2, job 3
    except fipc.FipcError as e:
        if e.result != fipc.Result.DISCONNECTED:  # the queue ended
            raise

Delivery. Each message goes to one consumer. It goes again, to another consumer, only if the process of the consumer that took it ends before acknowledging it, and when a consumer acknowledges is its own choice, not the queue's. A message received in place is acknowledged at its release (or the consumer's next receive, or its close).

• Choose by the work, per consumer. Work that may safely run twice (rendering, a computation, re-indexing, converting a file) acknowledges after it; work whose effects must never repeat (a payment, an email, a motor command) acknowledges before it.
• Not covered: a consumer that hangs while its process runs (there are no timeouts, by design); the owner's process, with which the queue ends; a reboot. A consumer holds at most one unacknowledged message at a time, the one received in place.
• A message that goes again comes in no set order, after messages its producer sent later: the queue promises no order anyway.
• A message held in place holds back its producer's room until its release: a consumer that acknowledges after a long job keeps that message's room, and so the room of the producer's messages after it, until then.
• A message that crashes every consumer that takes it, acknowledged after the work, goes to each consumer in turn, with no limit: acknowledge before the work where that can happen.

A service takes requests from up to 64 clients and hands each to one of up to 64 servers, in any processes of the user; the server that took a request replies to it, and the reply goes back to the client that sent it. A server waits for the requests of every client at once (one recv), and the servers of a pool share the load: each takes the next request of any client. Each client has a ring of its own, which holds its requests and the room for their replies, so a server never waits for a client. Requests and replies are of one piece, 1 to max_message bytes, the owner's choice. Here the server creates the service (service_create: a Service), joins it as its server (service_join with SERVER: the owner's process admits every joiner by itself, its own included) and answers three requests with their uppercase. The client joins (CLIENT) and calls ping 1 to ping 3: call sends a request and returns its reply. The service's close ends it: clients' sends and servers' receives raise DISCONNECTED, and a client receives the replies that came before the end, then DISCONNECTED; a client's or a server's close never ends the service. A Service never closes itself (it has no finalizer): a service lives until close() or the end of its process, and dropping the object doesn't end it.

import fipc

# 4 KiB per client, requests and replies of up to 256 bytes, 1 client and 1 server: this process owns the service
# and is its server
with fipc.service_create("my_service", 4096, 256, 1, 1):
    with fipc.service_join("my_service", fipc.SERVER, timeout_ms=5000) as server:
        for _ in range(3):  # three requests, then the end
            request = server.recv()  # the next request of any client
            server.send(request.upper())  # the reply goes to its client, and never waits
# closing the service ends it: the client still receives the replies sent

import fipc

with fipc.service_join("my_service", fipc.CLIENT, timeout_ms=5000) as client:
    for i in (1, 2, 3):
        # sends the request and returns its reply; FipcError (ABANDONED) if the server ended before it replied
        reply = client.call(f"ping {i}".encode())
        print(reply.decode())  # PING 1, PING 2, PING 3

• A client's replies come in the order of its requests. send(request) sends a request, and recv() (recv_into(buf), or in place recv_acquire() until recv_release()) receives the reply to its oldest request whose reply it hasn't received yet, whichever server replies first. A client may send several requests before it receives (as many as its ring has room for: each holds its length rounded up to 64 bytes, plus max_message rounded up to 64, until its reply has been received), and the servers work on them at once. A receive that times out or is cancelled leaves its reply to come: the next receive returns it.
• call(request) never returns another request's reply. A call checks its request first: INVALID for an empty one, TOO_LARGE over max_piece(), at once, even while a reply is owed. Then a call on a cancelled connection raises CANCELLED at once, whatever its timeout, and sends nothing: it could never receive the reply. The connection keeps count of the replies still owed: a call whose receive timed out or was cancelled leaves its reply owed, and the next call first receives and drops that reply (at most one is owed, since every call drops it first; an ABANDONED in its place counts too), within its own timeout and before it sends. A call that had to drop an owed reply first sends its request only if its timeout has time left: once a positive timeout has run out, it raises TIMEOUT without sending, so no server sees that request; with NO_WAIT (0) no step waits, and it sends if there is room. After a call that gave up, a plain recv returns that call's late reply (or ABANDONED) first: replies come in request order. A call raises INVALID while a request sent with send still has its reply to be received: that reply is yours, so receive it first. A call sends and receives, so no other thread may send or receive on the connection meanwhile.
• A server holds one request at a time. recv() takes the next request of any client (each client's in the order it sent them), and send(reply) replies to it and never waits; until then its receives raise INVALID, and a reply with no request held raises INVALID. A request received in place stays valid until the reply's send or send_commit; recv_release() does nothing. A reply to a client that has left is dropped. A server may receive on one thread and reply on another. Its close() answers the request it holds ABANDONED, and so does the garbage collection of a server's Conn left unclosed: a server holds its request, and its client waits for the reply, until it is closed (with with or close()) or garbage-collected (or its process ends).
• ABANDONED: a client's receive (or call) raises it when the server that took the request ended before it replied: it closed its connection, or its process ended. The request counts as answered, and the next receive returns the next reply. An abandoned request may or may not have been carried out; the service never sends it again, and the client decides whether to.
• The owner's service: service.wait_for_members(role, at_least) waits until that many members of the role (CLIENT, SERVER, or None for both) have joined; service.cancel() makes its waits raise CANCELLED, from any thread. Clients wait while no server is joined, and servers while no client is.
• Not covered: a server that hangs while its process runs (there are no timeouts, by design: its client waits, within its own timeouts); the owner's process, with which the service ends.

Measured through this binding, copied 16-byte messages run at about 1.2 to 1.5 million per second, and 64 KiB messages at about 320,000 to 340,000 per second (details).