FastIPC FastIPC v1.0.0
fipc on LuaRocks · require("fipc")

Lua

One module over LuaJIT's FFI, for LuaJIT 2.1 (also OpenResty's, Neovim's and LÖVE's): fipc.stream_create and fipc.stream_connect give listeners and connections with copied and zero-copy messages, fipc.broadcast_create and fipc.broadcast_join broadcasts, fipc.queue_create and fipc.queue_join work queues, fipc.service_create and fipc.service_join services. Nothing is compiled. A call that fails returns nil and the error's name, as Lua's io functions do, so assert() turns it into an error. The rock carries the native library for Windows x64, Linux x64 and ARM64 (glibc 2.34+) and macOS 14.4+ on Apple Silicon.

LuaRocksluarocks install fipc

Server and client

two processes, one name

Start the two in either order, in two terminals (luajit server.lua, luajit client.lua): fipc.stream_connect waits up to its timeout, in milliseconds, for a server to create the stream and accept; a call without a timeout waits as long as it takes. A Lua state has one thread, and stream_connect waits for the server's accept: in Lua, a client and its server run in different processes. The pair makes the exchange every example on this site makes, the message ping, answered in upper case: this client works with any language's server, and this server with any language's client.

server.lua
local fipc = require("fipc")

local listener = assert(fipc.stream_create("my_channel", 1024 * 1024)) -- rings of 1 MiB each way
local conn = assert(listener:accept()) -- waits for a client
local message = assert(conn:recv())
assert(conn:send(message:upper()))
conn:close()
listener:close()
client.lua
local fipc = require("fipc")

local conn = assert(fipc.stream_connect("my_channel", 5000)) -- waits up to 5 s for the server
assert(conn:send("ping"))
print(assert(conn:recv())) -- PING
conn:close()

Messages

bytes of any size, no header

A connection carries messages: send takes a message of any size, a Lua string (or a cdata pointer and its length), and recv returns each one whole as a Lua string. This server echoes every message back until its client leaves, which ends its next recv with "disconnected".

echo_server.lua
local fipc = require("fipc")

-- rings of 1 MiB each way: a power of two, 1 KiB to 2 GiB
local listener = assert(fipc.stream_create("demo", 1024 * 1024))
local conn = assert(listener:accept()) -- waits for a client
while true do
    local message, err = conn:recv()
    if message then
        assert(conn:send(message)) -- echo, any size
    elseif err == fipc.DISCONNECTED then
        break
    else
        error(err)
    end
end
print("client gone")
conn:close()
listener:close()
echo_client.lua
local fipc = require("fipc")

local conn = assert(fipc.stream_connect("demo", 5000))
for _, word in ipairs({ "hello", "shared", "memory" }) do
    assert(conn:send(word))
    print(assert(conn:recv()))
end
conn:close()

Errors and timeouts

nil and the error's name

A call that succeeds returns its value, or true when it has none. A call that doesn't returns nil and the error's name: "timeout", "disconnected", "cancelled", "too_large", "invalid", "no_memory", "addr_in_use", on a broadcast "overrun" or "unsubscribed", and on a service's client "abandoned" (the server that took the request ended before it replied), the C API's results (fipc.TIMEOUT and the others are the same strings; fipc.result_str(err) gives the C name). Wrap a call in assert() to raise instead. A misuse raises: a closed handle, an argument of the wrong type. The timeout comes last, in milliseconds: fipc.NO_WAIT (0) doesn't wait, nil or fipc.FOREVER waits as long as it takes. "disconnected" is final: close the connection, then accept or connect a new one.

wait at most 100 ms
local message, err = conn:recv(100) -- waits at most 100 ms
if message then
    handle(message)
elseif err ~= fipc.TIMEOUT then -- "timeout": nothing yet, try again later
    error(err) -- "disconnected": the peer is gone
end
a buffer of your own: "too_large" says how much room
local buf = ffi.new("uint8_t[?]", 4096)
local len, err, needed = conn:recv_into(buf, 4096)
if err == fipc.TOO_LARGE then -- the message stays queued: make room, then take it
    buf = ffi.new("uint8_t[?]", needed)
    len = assert(conn:recv_into(buf, needed))
end

Zero-copy

up to conn:max_piece() bytes

Write a message into the ring and commit it; read one in the ring and release it. send_acquire returns a uint8_t* into the ring, 16-byte aligned; send_commit publishes its first bytes (all, or fewer). recv_acquire returns a const uint8_t* and the length, valid until recv_release, the next receive or close: copy out what you keep. A message of several pieces is "too_large" for recv_acquire and stays queued for recv.

send in place
local slot = assert(conn:send_acquire(5)) -- room in the ring: a uint8_t*
ffi.copy(slot, "hello", 5)
assert(conn:send_commit(5))
receive in place
local data, len = assert(conn:recv_acquire()) -- a const uint8_t* into the ring
handle(data, len) -- read it in place
conn:recv_release() -- frees its room

Batches

many messages per call, both ends

conn:send_many(messages, timeout) sends a table of strings in order and publishes them together, with at most one wake of the receivers, and returns how many went; the timeout bounds the whole call (fipc.NO_WAIT: only what fits now). Every message is checked before any goes. A failure is the first message that didn't go, with how many went as the third value.

conn:recv_many(max_count, timeout) waits for the next message, then takes every further one already there (up to max_count, as many as fit the connection's buffer) and returns them as a table of strings. A broadcast's subscriber takes those its publisher has moved past; a service's client sends a batch of requests (owed as its sends are) and receives their replies in order; a server's batches return nil, "invalid". Batch both ends: a receiver that takes one message per call falls behind a batching sender.

a batch each way
local sent = assert(conn:send_many({ "one", "two", "three" })) -- published together
local batch = assert(conn:recv_many(64)) -- every message already there, up to 64
for _, message in ipairs(batch) do
    handle(message)
end

Broadcast

one publisher, many subscribers

A broadcast sends every message of one publisher to each of its subscribers, in any processes of the user; each message is stored once, in memory every subscriber maps. Messages are of one piece, up to a quarter of the capacity (max_piece()). fipc.broadcast_create(name, capacity, max_subscribers, max_stall) returns the publisher, a connection and the broadcast's one handle. A subscriber joins with fipc.broadcast_join, and the publisher's process admits it on a thread of the library's, so no call of the publisher's waits for it (the publisher's own Lua state may join too). The publisher waits for its subscriber (publisher:wait_for_members(1); at "timeout" or "cancelled" the count comes third, and fipc.NO_WAIT polls, once a frame) before it sends tick 1 to tick 3; closing the publisher ends the broadcast and frees its name. The subscriber joins gated (fipc.SUBSCRIBER_GATED), so it misses nothing (the publisher waits for it), and receives until "disconnected", which comes after the last message. A gated one that holds the publisher back for the broadcast's max_stall ms is unsubscribed ("unsubscribed"), and the memory of a message it holds in place (recv_acquire) becomes inaccessible, so reading it then crashes the process: release it well within max_stall, or create the broadcast with max_stall nil, which never unsubscribes.

publisher.lua
local fipc = require("fipc")

-- 64 KiB of messages, up to 8 subscribers; one that holds the publisher back 5 s is dropped
local publisher = assert(fipc.broadcast_create("my_broadcast", 64 * 1024, 8, 5000))
assert(publisher:wait_for_members(1)) -- waits for the first subscriber
for i = 1, 3 do
    assert(publisher:send("tick " .. i))
end
publisher:close() -- ends the broadcast: subscribers get "disconnected" after the ticks
subscriber.lua
local fipc = require("fipc")

-- gated: misses nothing (the publisher waits for it)
local subscriber = assert(fipc.broadcast_join("my_broadcast", fipc.SUBSCRIBER_GATED, 5000))
while true do
    local tick, err = subscriber:recv()
    if not tick then
        assert(err == fipc.DISCONNECTED, err) -- the publisher closed
        break
    end
    print(tick) -- tick 1, tick 2, tick 3
end
subscriber:close()

A lossy subscriber (fipc.SUBSCRIBER_LOSSY) never holds the publisher back, and receives copies only. One that falls a lap behind gets nil, "overrun" and the number of messages it lost (a Lua number, exact up to 253) instead of a message, and the next receive goes on from a recent message.

a lossy subscriber
local message, err, lost = subscriber:recv()
if message then
    handle(message)
elseif err == fipc.OVERRUN then -- fell a lap behind: the next receive goes on from a recent message
    print(lost .. " messages lost")
else
    error(err) -- "disconnected": the broadcast ended
end

Work queue

many producers, each message to one consumer

A work queue takes messages from up to 63 producers and hands each to one of up to 64 consumers, in any processes of the user. Each producer has a ring of its own; a consumer takes a message of any producer, and the queue promises no order. Messages are of one piece, up to a quarter of the capacity (max_piece()), and wait in the queue until a consumer takes them. Each message goes to one consumer, and again, to another, only if that consumer's process ends before acknowledging it (delivery, below). Here the producer owns the queue (fipc.queue_create returns it) and joins it as a producer (fipc.queue_join with fipc.PRODUCER): the owner's process admits every joiner on a thread of the library's, its own included, so one Lua state can own a queue and produce. It sends job 1 to job 3, which wait in the queue, then waits for the consumer (queue:wait_for_members(fipc.CONSUMER, 1); a role of nil counts both), so the two start in either order. Closing the queue ends it: the consumer receives what is left, then "disconnected", and a producer's sends return nil, "disconnected" from then on; a producer's or a consumer's close never ends it. The garbage collector never closes a queue: a queue lives until close() or the end of its process, and dropping the object doesn't end it. A consumer receives copied or in place (recv_acquire).

producer.lua
local fipc = require("fipc")

-- 4 KiB of messages per producer, 1 producer and 1 consumer: this process owns the queue and is its producer
local queue = assert(fipc.queue_create("my_queue", 4096, 1, 1))
local producer = assert(fipc.queue_join("my_queue", fipc.PRODUCER))
for i = 1, 3 do
    assert(producer:send("job " .. i)) -- waits in the queue until a consumer takes it
end
assert(queue:wait_for_members(fipc.CONSUMER, 1)) -- waits for the consumer to join
producer:close()
queue:close() -- ends the queue: the consumer gets "disconnected" after the jobs
consumer.lua
local fipc = require("fipc")

local consumer = assert(fipc.queue_join("my_queue", fipc.CONSUMER, 5000))
while true do
    local job, err = consumer:recv()
    if not job then
        assert(err == fipc.DISCONNECTED, err) -- the queue ended
        break
    end
    print(job) -- job 1, job 2, job 3
end
consumer:close()

Delivery. Each message goes to one consumer. It goes again, to another consumer, only if the process of the consumer that took it ends before acknowledging it, and when a consumer acknowledges is its own choice, not the queue's. A message received in place is acknowledged at its release (or the consumer's next receive, or its close).

How the consumer receivesWhen it acknowledgesIf its process endsWhat the application gets
recv() or recv_into() (copying)inside the call, before it returnsinside the call: the message goes to another consumer, and this one never saw it; after the call returned: it doesn't go againat most once: no duplicates
recv_acquire(), then recv_release() at once, then the workat the release, before the workas aboveat most once: no duplicates
recv_acquire(), the work, then recv_release()at the release, after the workbefore the release: the message goes to another consumerat least once: a duplicate is possible, so the work must be idempotent

Service

many clients, a pool of servers, with replies

A service takes requests from up to 64 clients and hands each to one of up to 64 servers, in any processes of the user; the server that took a request replies to it, and the reply goes back to the client that sent it. A server waits for the requests of every client at once (one recv), and the servers of a pool share the load: each takes the next request of any client. Each client has a ring of its own, which holds its requests and the room for their replies, so a server never waits for a client. Requests and replies are of one piece, 1 to max_message bytes, the owner's choice (max_piece()). Here the server creates the service (fipc.service_create returns it), joins it as its server (fipc.service_join with fipc.SERVER: the owner's process admits every joiner by itself, its own included, so one Lua state can own a service and serve) and answers three requests with their uppercase. The client joins (fipc.CLIENT) and calls ping 1 to ping 3: call sends a request and returns its reply. The service's close() ends it: clients' sends and servers' receives return nil, "disconnected", and a client receives the replies that came before the end, then "disconnected"; a client's or a server's close never ends the service. The garbage collector never closes a service: a service lives until close() or the end of its process, and dropping the object doesn't end it.

service_server.lua
local fipc = require("fipc")

-- 4 KiB per client, requests and replies of up to 256 bytes, 1 client and 1 server: this process owns the service
-- and is its server
local service = assert(fipc.service_create("my_service", 4096, 256, 1, 1))
local server = assert(fipc.service_join("my_service", fipc.SERVER, 5000))
for _ = 1, 3 do -- three requests, then the end
    local request = assert(server:recv()) -- the next request of any client
    assert(server:send(request:upper())) -- the reply goes to its client, and never waits
end
server:close()
service:close() -- ends the service: the client still receives the replies sent
service_client.lua
local fipc = require("fipc")

local client = assert(fipc.service_join("my_service", fipc.CLIENT, 5000))
for i = 1, 3 do
    -- sends the request and returns its reply; nil, "abandoned" if the server ended before it replied
    print(assert(client:call("ping " .. i))) -- PING 1, PING 2, PING 3
end
client:close()

Any language on the other end

one protocol

The server.lua and client.lua above work with a peer in any of the ten languages: every pair of languages is tested.

Threads and coroutines

one Lua state, one thread

A Lua state runs one thread at a time, so the C API's rule (on a connection, one thread sends and one receives) holds by itself. A call that waits blocks the whole state, every coroutine in it: in a coroutine scheduler, poll with fipc.NO_WAIT and yield while nothing has arrived. In OpenResty a call that waits blocks the nginx worker, so poll and ngx.sleep between polls; in Neovim, poll from a vim.uv timer.

conn:cancel() makes every call of the connection that waits, then or later, return "cancelled". A handle belongs to the Lua state that made it; another state (another thread: love.thread, Lanes) opens a connection of its own. close() ends a connection at once; one the program drops is closed when the garbage collector frees it. So is a dropped listener; a queue or a service is not: it lives until close() or the end of its process.

yield until a message comes
-- In a coroutine scheduler: poll, and yield until a message (or an error) comes
local function receive(conn)
    while true do
        local message, err = conn:recv(fipc.NO_WAIT)
        if message or err ~= fipc.TIMEOUT then
            return message, err
        end
        coroutine.yield()
    end
end

Games and frame loops

never block a frame

Poll the connection once a frame with fipc.NO_WAIT: send what the frame produced, then take every message that has arrived until "timeout" says there is none. In LÖVE, do it in love.update; a game that embeds LuaJIT does it in its own update. A send with NO_WAIT returns "timeout" only when the ring is full.

game_loop.lua runs against echo_server.lua above, sleeping for the rest of each frame as an engine's loop would.

game_loop.lua
local ffi = require("ffi")
local fipc = require("fipc")

-- The rest of a frame (a game engine's loop does this for you)
local sleep
if ffi.os == "Windows" then
    ffi.cdef("void Sleep(uint32_t ms);")
    sleep = function(ms) ffi.C.Sleep(ms) end
else
    ffi.cdef("int usleep(uint32_t us);")
    sleep = function(ms) ffi.C.usleep(ms * 1000) end
end

local conn = assert(fipc.stream_connect("demo", 5000))
local echoes = 0
for frame = 1, 60 do
    assert(conn:send(tostring(frame), fipc.NO_WAIT)) -- "timeout" only if the ring is full
    while true do
        local echo, err = conn:recv(fipc.NO_WAIT)
        if echo then
            echoes = echoes + 1
        elseif err == fipc.TIMEOUT then
            break -- nothing more this frame
        else
            error(err) -- "disconnected": the peer is gone
        end
    end
    sleep(16)
end
print(echoes .. " echoes in 60 frames")
conn:close()

Install and ship

LuaJIT 2.1 · x86-64-v3 · ARM64 Linux · Apple Silicon

The rock installs the module and the library for your platform; LuaRocks must be set up for LuaJIT (--lua-version 5.1, --lua-dir). PUC Lua (5.1 to 5.4) has no FFI and isn't supported. Without LuaRocks, ship fipc.lua with fastipc.dll, libfastipc.so or libfastipc.dylib beside it, in a folder on package.path: the GitHub release's fipc-lua-1.0.0.tar.gz holds the module and every platform's library.

The module looks for the library in the folder FASTIPC_LIB_DIR names, then (in a checkout of the repository) its zig-out build, then its own folder, then the rock's copy on package.cpath, then the system's search. fipc.library says which file it loaded; the library stays loaded until the process ends.

with LuaRocks, for LuaJIT
luarocks --lua-version 5.1 install fipc
luajit -e 'print(require("fipc").library)'
shipped next to your code
game/
  main.lua
  fipc.lua
  fastipc.dll        (Windows; libfastipc.so on Linux, libfastipc.dylib on macOS)

Reference

fipc.lua
CallWhat it does
fipc.stream_create(name, capacity)Claims the name for a stream and listens: a listener; "addr_in_use" if another stream, broadcast, queue or service holds it.
listener:accept(timeout)Sets the next client's connection up and returns it. One client at a time: close it before the next accept. fipc.NO_WAIT polls.
listener:cancel() · close()Cancel waiting accepts · stop listening and free the name; accepted connections stay open.
fipc.stream_connect(name, timeout)Connects to the stream's server, waiting for it to create the stream and accept: a connection. The server runs in another process.
conn:send(data, timeout)Sends one message of any size: a string, or ptr, len. Returns true.
conn:recv(timeout)Receives one message of any size, as a string.
conn:recv_into(buf, size, timeout)Receives into your buffer and returns the length; nil, "too_large", len if it doesn't fit.
conn:send_many(messages, timeout) · recv_many(max_count, timeout)Batches: send a table of strings, published together, and return how many went (on a failure, the third value) · receive every message already there, up to max_count, as a table. Batch both ends.
conn:send_acquire(len, timeout) · send_commit(len)Zero-copy send: a uint8_t* into the ring, then publish.
conn:recv_acquire(timeout) · recv_release()Zero-copy receive: a const uint8_t* and the length, then free its room.
conn:max_piece() · cancel() · ended() · close()The zero-copy limit · cancel waiting calls · whether a call returned "disconnected" · end the connection.
fipc.broadcast_create(name, capacity, max_subscribers, max_stall)Claims the name for a broadcast: the publisher, a connection whose sends go to every joined subscriber. Closing it ends the broadcast.
publisher:wait_for_members(at_least, timeout)The number of subscribers joined, once at least at_least are; nil, "timeout", count.
fipc.broadcast_join(name, role, timeout)Joins the broadcast as a fipc.SUBSCRIBER_GATED or fipc.SUBSCRIBER_LOSSY subscriber, waiting for the publisher's process to admit it: a connection that receives; nil, "overrun", lost.
fipc.queue_create(name, capacity, max_producers, max_consumers)Claims the name for a work queue: the queue. It lives until close() or the end of its process; the garbage collector never closes it.
queue:wait_for_members(role, at_least, timeout)The number of members of the role joined (nil: both roles), once at least at_least are; nil, "timeout", count.
queue:cancel() · close()Cancel waiting wait_for_members calls · end the queue: producers get "disconnected", consumers what is left, then "disconnected".
fipc.queue_join(name, role, timeout)Joins the queue as a fipc.PRODUCER, which sends, or a fipc.CONSUMER, which receives each message no other consumer took, then "disconnected" once the queue has ended; the owner's members too.
fipc.service_create(name, capacity, max_message, max_clients, max_servers)Claims the name for a service: the service. It lives until close() or the end of its process; the garbage collector never closes it.
service:wait_for_members(role, at_least, timeout)The number of members of the role joined (nil: both roles), once at least at_least are; nil, "timeout", count.
service:cancel() · close()Cancel waiting wait_for_members calls · end the service: clients' sends and servers' receives get "disconnected", clients the replies that came first, then "disconnected".
fipc.service_join(name, role, timeout)Joins the service as a fipc.CLIENT, whose sends are requests and whose receives their replies, in order, or a fipc.SERVER, which receives the next request of any client and replies with a send; the owner's members too.
client:call(request, timeout)Sends a request and returns its reply; nil, "abandoned" if its server ended first. A call never returns another request's reply.
fipc.NO_WAIT · FOREVER · SUBSCRIBER_LOSSY · SUBSCRIBER_GATED · PRODUCER · CONSUMER · CLIENT · SERVER0 · -1 · 1 · 2 · 3 · 4 · 5 · 6.
fipc.TIMEOUT … fipc.ABANDONED · result_str(err)The error names · the C name of one ("FIPC_TIMEOUT").
fipc.library · fipc.C · fipc._VERSIONThe library's path · its FFI namespace, the C API · "1.0.0".

Measured through this binding on LuaJIT, copied 16-byte messages run at 49 to 51 million per second between two processes, and 64 KiB messages at 400,000 to 430,000 per second (details).