Rust
A safe API, the modules stream, broadcast, queue and service with Listener, Connection, Queue and Service, over the raw FFI crate fipc-sys, for Rust 1.85 and later, with no other dependencies. Calls return Result<T, Error>; handles close when dropped; the borrow checker keeps the C API's rules on threads and zero-copy buffers. The -sys crate carries the native library for Windows x64, Linux x64 and ARM64 (glibc 2.34+) and macOS 14.4+ on Apple Silicon, and links it.
cargo add fipcServer and client
two processes, one nameStart the two in either order, in two terminals: fipc::stream::connect waits up to its timeout for a server to create the stream and accept. Each file is a program of its own (src/main.rs, or an example: cargo run --example server). A timeout is anything that converts into an Option<Duration>: None waits as long as it takes. The pair makes the exchange every example on this site makes, the message ping, answered in upper case: this client works with any language's server, and this server with any language's client.
fn main() -> fipc::Result<()> {
let mut listener = fipc::stream::create("my_channel", 1 << 20)?; // rings of 1 MiB each way
let mut conn = listener.accept(None)?; // waits for a client
let message = conn.receive(None)?;
conn.send(&message.to_ascii_uppercase(), None)?;
Ok(())
}use std::time::Duration;
fn main() -> fipc::Result<()> {
let mut conn = fipc::stream::connect("my_channel", Duration::from_secs(5))?;
conn.send(b"ping", None)?;
println!("{}", String::from_utf8_lossy(&conn.receive(None)?)); // PING
Ok(())
}Messages
bytes of any size, no headerA connection carries messages: send takes a message of any size, and receive returns each one whole in a new Vec (or fills your buffer). This server echoes every message back until its client leaves, which ends its next receive with Error::Disconnected.
use fipc::Error;
fn main() -> fipc::Result<()> {
// rings of 1 MiB each way: a power of two, 1 KiB to 2 GiB
let mut listener = fipc::stream::create("demo", 1 << 20)?;
let mut conn = listener.accept(None)?; // waits for a client
loop {
match conn.receive(None) {
Ok(message) => conn.send(&message, None)?, // echo, any size
Err(Error::Disconnected) => break,
Err(e) => return Err(e),
}
}
println!("client gone");
Ok(())
}use std::time::Duration;
fn main() -> fipc::Result<()> {
let mut conn = fipc::stream::connect("demo", Duration::from_secs(5))?;
for word in ["hello", "shared", "memory"] {
conn.send(word.as_bytes(), None)?;
println!("{}", String::from_utf8_lossy(&conn.receive(None)?));
}
Ok(())
}Errors and timeouts
fipc::Error mirrors fipc_result_tEvery call that can fail returns fipc::Result<T>. Error is an enum: Timeout, Disconnected, Cancelled, TooLarge { len } (the room a receive needs; the message stays queued), Invalid, NoMemory or AddrInUse, on a broadcast Overrun { lost } (a lossy subscriber lost messages) or Unsubscribed, and on a service's client Abandoned (the server that took the request ended before it replied). It implements std::error::Error, displays the C name (FIPC_TIMEOUT: the timeout ran out) and converts into std::io::Error.
The timeout comes last: a Duration, NO_WAIT (doesn't wait), or None / FOREVER. Disconnected is final for that connection: drop it, then accept, connect or join anew.
match conn.receive(Duration::from_millis(100)) {
Ok(message) => handle(&message),
Err(Error::Timeout) => {} // nothing yet
Err(e) => return Err(e), // Error::Disconnected: the peer is gone
}Zero-copy
up to conn.max_piece() bytesWrite into a SendSlot of the ring and commit it; read a ReceiveSlot of the ring, which frees its room when dropped. Both deref to [u8]. A slot borrows its side of the connection, so the borrow checker keeps it from outliving its turn: nothing else is sent (or received) while it lives. Dropping a SendSlot without committing it sends nothing.
let mut slot = conn.send_acquire(5, None)?; // room in the ring
slot.copy_from_slice(b"hello");
slot.commit(5)?;let message = conn.receive_acquire(None)?; // read-only, in the ring
handle(&message);
drop(message); // frees its roomBatches
many messages per call, both endssend_many(&messages, timeout) sends a slice of messages (anything AsRef<[u8]>) in order and publishes them together, with at most one wake of the receivers, and returns how many went; the timeout bounds the whole call (NO_WAIT: only what fits now). Every message is checked before any goes. Its error, a SendManyError, holds the first message's error and how many were sent.
receive_many(&mut buf, &mut lens, timeout) waits for the next message, then takes every further one already there that fits buf, packed back to back, their lengths in lens, and returns how many; fipc::batch_messages walks them. Any error is the first message's, as receive_into returns it. A broadcast's subscriber takes those its publisher has moved past; a service's client sends a batch of requests (owed as its sends are) and receives their replies in order; a server's batches return Error::Invalid. Batch both ends: a receiver that takes one message per call falls behind a batching sender.
let batch: [&[u8]; 3] = [b"one", b"two", b"three"];
let sent = conn.send_many(&batch, None)?; // published together, at most one wake
let mut buf = vec![0; 1 << 16];
let mut lens = [0; 64];
let count = peer.receive_many(&mut buf, &mut lens, None)?; // every message already there
for message in fipc::batch_messages(&buf, &lens[..count]) {
handle(message);
}Broadcast
one publisher, many subscribersA broadcast sends every message of one publisher to each of its subscribers, in any processes of the user; each message is stored once, in memory every subscriber maps. Messages are of one piece, up to a quarter of the capacity (max_piece()). fipc::broadcast::create returns the publisher, a Connection and the broadcast's one handle. The publisher's process admits each subscriber that joins on a thread of the library's; the publisher waits for the first one (publisher.wait_for_members(1, None), which takes &self; at a timeout the MembersError's count is the number joined) before it sends tick 1 to tick 3; dropping the publisher ends the broadcast and frees the name. The subscriber joins gated (fipc::broadcast::join with SubscriberRole::Gated), so it misses nothing (the publisher waits for it), may receive in place with the unsafe receive_acquire_gated (its memory can vanish once it is unsubscribed: drop the slot before the broadcast's max_stall, or create it with None), and receives until Error::Disconnected, which comes after the last message. A gated one that holds the publisher back for the broadcast's max_stall is unsubscribed (Error::Unsubscribed).
use std::time::Duration;
fn main() -> fipc::Result<()> {
// 64 KiB of messages, up to 8 subscribers; one that holds the publisher back 5 s is dropped
let mut publisher = fipc::broadcast::create("my_broadcast", 1 << 16, 8, Duration::from_secs(5))?;
publisher.wait_for_members(1, None)?; // waits for the first subscriber
for i in 1..=3 {
publisher.send(format!("tick {i}").as_bytes(), None)?;
}
Ok(())
} // dropping the publisher ends the broadcast: subscribers get Error::Disconnected after the ticksuse fipc::{Error, SubscriberRole};
use std::time::Duration;
fn main() -> fipc::Result<()> {
// gated: misses nothing (the publisher waits for it)
let mut subscriber = fipc::broadcast::join("my_broadcast", SubscriberRole::Gated, Duration::from_secs(5))?;
loop {
match subscriber.receive(None) {
Ok(tick) => println!("{}", String::from_utf8_lossy(&tick)), // tick 1, tick 2, tick 3
Err(Error::Disconnected) => return Ok(()), // the publisher closed
Err(e) => return Err(e),
}
}
}A lossy subscriber (SubscriberRole::Lossy) never holds the publisher back, and receives copies only. One that falls a lap behind gets Error::Overrun { lost } instead of a message, lost (a u64) being the number of messages it lost, and the next receive goes on from a recent message.
match subscriber.receive(None) {
Ok(message) => handle(&message),
// Fell a lap behind: the next receive goes on from a recent message
Err(Error::Overrun { lost }) => println!("{lost} messages lost"),
Err(e) => return Err(e), // Error::Disconnected: the broadcast ended
}Work queue
many producers, each message to one consumerA work queue takes messages from up to 63 producers and hands each to one of up to 64 consumers, in any processes of the user. Each producer has a ring of its own; a consumer takes a message of any producer: the queue promises no order. Messages are of one piece, up to a quarter of the capacity (max_piece()), and wait in the queue until a consumer takes them. Each message goes to one consumer, and again, to another, only if that consumer's process ends before acknowledging it (delivery, below). fipc::queue::create returns the owner's Queue; producers and consumers join with fipc::queue::join (QueueRole::Producer, QueueRole::Consumer), the owner too, and the owner's process admits each one on a thread of the library's. Here the owner joins as the producer, sends job 1 to job 3, which wait in the queue, and waits for the consumer (queue.wait_for_members(QueueRole::Consumer, 1, None)). A queue lives until its Queue is closed or dropped, or the owner's process ends: then the consumer receives what is left, then Error::Disconnected, and a producer's sends return Error::Disconnected. queue.close() or the drop ends it at a place you see; nothing else does (no garbage collector): binding it to _, or not at all, drops it, and the queue ends at once. A producer's or a consumer's drop never ends the queue. A consumer receives copied or in place (receive_acquire).
use fipc::QueueRole;
use std::time::Duration;
fn main() -> fipc::Result<()> {
// 4 KiB of messages per producer, 1 producer and 1 consumer: this process owns the queue and is its producer
let queue = fipc::queue::create("my_queue", 4096, 1, 1)?;
let mut producer = fipc::queue::join("my_queue", QueueRole::Producer, Duration::from_secs(5))?;
for i in 1..=3 {
producer.send(format!("job {i}").as_bytes(), None)?; // waits in the queue until a consumer takes it
}
queue.wait_for_members(QueueRole::Consumer, 1, None)?; // waits for the consumer to join
Ok(())
} // dropping the producer, then the queue, ends the queue: the consumer gets Error::Disconnected after the jobsuse fipc::{Error, QueueRole};
use std::time::Duration;
fn main() -> fipc::Result<()> {
let mut consumer = fipc::queue::join("my_queue", QueueRole::Consumer, Duration::from_secs(5))?;
loop {
match consumer.receive(None) {
Ok(job) => println!("{}", String::from_utf8_lossy(&job)), // job 1, job 2, job 3
Err(Error::Disconnected) => return Ok(()), // the queue ended
Err(e) => return Err(e),
}
}
}Delivery. Each message goes to one consumer. It goes again, to another consumer, only if the process of the consumer that took it ends before acknowledging it, and when a consumer acknowledges is its own choice, not the queue's. A message received in place is acknowledged at its release (or the consumer's next receive, or its close).
| How the consumer receives | When it acknowledges | If its process ends | What the application gets |
|---|---|---|---|
receive() or receive_into() (copying) | inside the call, before it returns | inside the call: the message goes to another consumer, and this one never saw it; after the call returned: it doesn't go again | at most once: no duplicates |
receive_acquire(), then the slot's release() (or its drop) at once, then the work | at the release, before the work | as above | at most once: no duplicates |
receive_acquire(), the work, then the slot's release() (or its drop) | at the release, after the work | before the release: the message goes to another consumer | at least once: a duplicate is possible, so the work must be idempotent |
- Choose by the work, per consumer. Work that may safely run twice (rendering, a computation, re-indexing, converting a file) acknowledges after it; work whose effects must never repeat (a payment, an email, a motor command) acknowledges before it.
- Not covered: a consumer that hangs while its process runs (there are no timeouts, by design); the owner's process, with which the queue ends; a reboot. A consumer holds at most one unacknowledged message at a time, the one received in place.
- A message that goes again comes in no set order, after messages its producer sent later: the queue promises no order anyway.
- A message held in place holds back its producer's room until its release: a consumer that acknowledges after a long job keeps that message's room, and so the room of the producer's messages after it, until then.
- A message that crashes every consumer that takes it, acknowledged after the work, goes to each consumer in turn, with no limit: acknowledge before the work where that can happen.
Service
many clients, a pool of servers, with repliesA service takes requests from up to 64 clients and hands each to one of up to 64 servers, in any processes of the user; the server that took a request replies to it, and the reply goes back to the client that sent it. A server waits for the requests of every client at once (one receive), and the servers of a pool share the load: each takes the next request of any client. Each client has a ring of its own, which holds its requests and the room for their replies, so a server never waits for a client. Requests and replies are of one piece, 1 to max_message bytes, the owner's choice (max_piece()). Here the server creates the service (fipc::service::create returns the owner's Service), joins it as its server (fipc::service::join with ServiceRole::Server: the owner's process admits every joiner on a thread of the library's, its own included) and answers three requests with their uppercase. The client joins (ServiceRole::Client) and calls ping 1 to ping 3: call sends a request and returns its reply. A service lives until its Service is closed or dropped, or the owner's process ends: then clients' sends and servers' receives return Error::Disconnected, and a client receives the replies that came before the end, then Error::Disconnected. service.close() or the drop ends it at a place you see; nothing else does: binding it to _, or not at all, drops it, and the service ends at once. A client's or a server's drop never ends the service.
use fipc::ServiceRole;
use std::time::Duration;
fn main() -> fipc::Result<()> {
// 4 KiB per client, requests and replies of up to 256 bytes, 1 client and 1 server: this process owns the
// service and is its server
let service = fipc::service::create("my_service", 4096, 256, 1, 1)?;
let mut server = fipc::service::join("my_service", ServiceRole::Server, Duration::from_secs(5))?;
for _ in 0..3 {
// three requests, then the end
let request = server.receive(None)?; // the next request of any client
server.send(&request.to_ascii_uppercase(), None)?; // the reply goes to its client, and never waits
}
service.close(); // ends the service: the client still receives the replies sent
Ok(())
}use fipc::ServiceRole;
use std::time::Duration;
fn main() -> fipc::Result<()> {
let mut client = fipc::service::join("my_service", ServiceRole::Client, Duration::from_secs(5))?;
for i in 1..=3 {
// sends the request and returns its reply; Error::Abandoned if the server ended before it replied
let reply = client.call(format!("ping {i}").as_bytes(), None)?;
println!("{}", String::from_utf8_lossy(&reply)); // PING 1, PING 2, PING 3
}
Ok(())
}- A client's replies come in the order of its requests.
send(request)sends a request (orsend_acquireand the slot'scommit), and a receive (receive,receive_into, or in placereceive_acquire) returns the reply to its oldest request whose reply it hasn't received yet, whichever server replies first. A client may send several requests before it receives (as many as its ring has room for: each holds its length rounded up to 64 bytes, plusmax_messagerounded up to 64, until its reply has been received), and the servers work on them at once. A receive that times out or is cancelled leaves its reply to come: the next receive returns it. call(request, timeout)never returns another request's reply. The connection keeps count of the replies still owed: a call that timed out or was cancelled waiting for its reply leaves that reply owed, and the next call first receives and drops that reply (at most one is owed, since every call drops it first; anError::Abandonedin its place counts too), within its own timeout and before it sends. A call checks its request first:Error::Invalidfor an empty one,Error::TooLargeovermax_piece(), at once, even while a reply is owed. Then a call on a cancelled connection returnsError::Cancelledat once, whatever its timeout, and sends nothing: it could never receive the reply. A call that had to drop an owed reply first sends its request only if its timeout has time left: once a positive timeout has run out, it returnsError::Timeoutwithout sending, so no server sees that request; with a zero timeout no step waits, and it sends if there is room. After a call that gave up, a plainreceivereturns that call's late reply (orError::Abandoned) first: replies come in request order. A call returnsError::Invalidwhile a request sent withsend(or a slot'scommit) still has its reply to be received: that reply is yours, so receive it first; on any connection other than a service's client it returnsError::Invalidtoo. It takes&mut self: it sends and receives, and the borrow checker keeps another thread from doing either meanwhile.- A server holds one request at a time.
receivetakes the next request of any client (each client's in the order it sent them), andsend(reply)replies to it and never waits; until then its receives returnError::Invalid, and a reply with no request held returnsError::Invalid. A reply to a client that has left is dropped. A server may receive on one thread and reply on another (into_split()), and receives its next request once it has replied. - A request in place is read before its reply, never after. From the reply on, its bytes are its client's again, and the borrow checker keeps them from being read:
receive_request()returns aRequest, which derefs to the request in the ring, borrows the whole connection, and is consumed by its reply,reply(&[u8])(a copy) orreply_in_place(len, write), which handswritethe request and room for the reply in the ring, apart from the request's. ARequestisSend: another thread may reply. AReceiveSlotfromreceive_acquire()borrows the whole connection too, so the reply waits for its drop; a server's splitReceiverrefusesreceive_acquire()(Error::Invalid). Error::Abandoned: a client's receive (or call) returns it when the server that took the request ended before it replied: it was dropped, or its process ended. The request counts as answered, and the next receive returns the next reply. An abandoned request may or may not have been carried out; the service never sends it again, and the client decides whether to.- The owner's service:
service.wait_for_members(role, at_least, timeout)waits until that many members of the role (ServiceRole::Client,ServiceRole::Server, orNonefor both) have joined, and returns the count, or aMembersError;service.cancel()or aservice.canceler()ends such a wait from any thread. Clients wait while no server is joined, and servers while no client is. - Not covered: a server that hangs while its process runs (there are no timeouts, by design: its client waits, within its own timeouts); the owner's process, with which the service ends.
let request = server.receive_request(None)?; // held in place until the reply
let len = request.len();
request.reply_in_place(len, |request, reply| {
reply.copy_from_slice(request); // the reply's room, apart from the request: zero-copy both ways
reply.make_ascii_uppercase();
len
})?;match client.call(b"charge 42", Duration::from_secs(5)) {
Ok(reply) => handle(&reply),
// Its server ended before it replied: the charge may or may not have been made
Err(Error::Abandoned) => println!("ask the ledger before trying again"),
Err(e) => return Err(e), // Error::Disconnected: the service ended
}Any language on the other end
one protocolThe server.rs and client.rs above work with a peer in any of the ten languages: every pair of languages is tested.
Threads
Send + Sync · &mut self · CancelerListener, Connection and its halves, Queue and Service are Send and Sync. Calls that send or receive take &mut self, which is the C API's rule: one thread at a time sends, one at a time receives. To do both at once on two threads, split() the connection into its Sender and Receiver (borrowed) or into_split() it (owned); it closes when the last half is dropped.
To stop a thread that waits, take a Canceler first and call cancel() from any thread: every call that waits, then or later, returns Error::Cancelled. Then join the thread and drop the handle. A canceler doesn't keep its handle open.
let canceler = conn.canceler();
let reader = std::thread::spawn(move || {
// until Error::Cancelled (or Error::Disconnected)
while let Ok(message) = conn.receive(None) {
handle(&message);
}
});
// ...
canceler.cancel(); // wakes the reader: Cancelled
reader.join().unwrap();Games and frame loops
never block a framePoll the connection once a frame with NO_WAIT, or give its Receiver to a thread that forwards messages through a channel the frame loop drains. In Bevy, a Connection (or the channel's receiving end) fits in a Resource and is polled in a system. game_loop.rs runs against echo_server.rs.
use fipc::{Error, NO_WAIT};
use std::time::Duration;
fn main() -> fipc::Result<()> {
let mut conn = fipc::stream::connect("demo", Duration::from_secs(5))?;
let mut echoes = 0;
for frame in 0..60u32 {
conn.send(&frame.to_le_bytes(), NO_WAIT)?; // Error::Timeout only if the ring is full
loop {
match conn.receive(NO_WAIT) {
Ok(_echo) => echoes += 1,
Err(Error::Timeout) => break, // nothing more this frame
Err(e) => return Err(e), // Error::Disconnected: the peer is gone
}
}
std::thread::sleep(Duration::from_millis(16)); // the rest of the frame
}
println!("{echoes} echoes in 60 frames");
Ok(())
}// Sending on this thread, receiving on another: the halves of the connection
let (mut sender, mut receiver) = conn.into_split();
let reader = std::thread::spawn(move || {
while let Ok(message) = receiver.receive(None) {
if tx.send(message).is_err() {
break; // the frame loop is gone
}
}
});
// Each frame: whatever has arrived, without waiting
sender.send(b"update", None)?;Install and ship
Rust 1.85+ · x86-64-v3 · ARM64 Linux · Apple Siliconfipc-sys links the library dynamically (on Windows through raw-dylib: no import library) and copies it into Cargo's build folder, where cargo run and cargo test find it. It takes the library from the folder FASTIPC_LIB_DIR names, else its own copy. On docs.rs nothing links.
A program started on its own needs the library beside it: on Windows, fastipc.dll next to the .exe; on Linux, libfastipc.so next to the executable, linked with an rpath of $ORIGIN; on macOS, libfastipc.dylib next to the executable, linked with an rpath of @loader_path. After a build the file is in target/<profile>/build/fipc-sys-*/out/; a build script of a crate that depends on fipc-sys gets that folder as DEP_FASTIPC_LIB_DIR.
[dependencies]
fipc = "1.0"[target.x86_64-unknown-linux-gnu]
rustflags = ["-C", "link-arg=-Wl,-rpath,$ORIGIN"]
[target.aarch64-unknown-linux-gnu]
rustflags = ["-C", "link-arg=-Wl,-rpath,$ORIGIN"]
[target.aarch64-apple-darwin]
rustflags = ["-C", "link-arg=-Wl,-rpath,@loader_path"]Reference
docs.rs/fipc| Call | What it does |
|---|---|
fipc::stream::create(name, capacity) | Claims the name for a stream and returns its Listener; AddrInUse if another stream, broadcast, queue or service holds it. |
listener.accept(timeout) | Sets the next client's connection up on the calling thread and returns its Connection. One client at a time: drop it before the next accept. NO_WAIT polls. |
listener.cancel() · canceler() | Cancel waiting accepts · a Canceler that does it from any thread. Dropping the listener stops listening; accepted connections stay open. |
fipc::stream::connect(name, timeout) | Connects to the stream's server, waiting for it to create the stream and accept: client and server run on different threads or processes. |
send(&[u8], timeout) | Sends one message of any size. |
receive(timeout) | Receives one message of any size as a new Vec<u8>. |
receive_into(&mut [u8], timeout) | Receives into your buffer and returns the length; TooLarge { len } if it doesn't fit. |
send_acquire(len, timeout) · slot.commit(len) | Zero-copy send. |
receive_acquire(timeout) · drop the slot | Zero-copy receive. |
split() · into_split() | The Sender and Receiver halves, borrowed · owned, for two threads. |
max_piece() · cancel() · canceler() | The zero-copy limit · cancel waiting calls · a Canceler for any thread. Dropping the connection ends it. |
fipc::broadcast::create(name, capacity, max_subscribers, max_stall) | Claims the name for a broadcast; returns its publisher, a Connection whose sends go to every subscriber. Dropping it ends the broadcast. |
publisher.wait_for_members(at_least, timeout) | Waits until at least that many subscribers are joined and returns the count; a MembersError carries the count at a timeout. Any thread (&self). |
fipc::broadcast::join(name, role, timeout) | Joins the broadcast as a SubscriberRole::Gated or SubscriberRole::Lossy subscriber, waiting for the publisher's process to admit it; it receives, then Disconnected once the publisher is dropped. |
fipc::queue::create(name, capacity, max_producers, max_consumers) | Claims the name for a work queue; returns its Queue. Closing or dropping the Queue ends the queue. |
fipc::queue::join(name, role, timeout) | Joins the queue as a QueueRole::Producer, which sends, or a QueueRole::Consumer, which receives each message no other consumer took, then Disconnected once the queue has ended. |
queue.wait_for_members(role, at_least, timeout) | Waits until at least that many members of the role (None: both) are joined and returns the count, or a MembersError. Any thread (&self). |
queue.cancel() · canceler() · close() | Cancel waiting counts · a Canceler for any thread · end the queue (as its drop does). |
fipc::service::create(name, capacity, max_message, max_clients, max_servers) | Claims the name for a service; returns its Service. Closing or dropping the Service ends the service. |
fipc::service::join(name, role, timeout) | Joins the service as a ServiceRole::Client, which sends requests and receives their replies in order, or a ServiceRole::Server, which receives the next request of any client and replies with its next send. |
call(&[u8], timeout) | A service's client: sends the request and returns its reply as a new Vec<u8>; Abandoned if its server ended before it replied. |
receive_request(timeout) · request.reply(&[u8]) · reply_in_place(len, write) | A service's server: the next request, held in place as a Request · its reply, copied · its reply written in the ring. The reply consumes the request. |
service.wait_for_members(role, at_least, timeout) | Waits until at least that many members of the role (None: both) are joined and returns the count, or a MembersError. Any thread (&self). |
service.cancel() · canceler() · close() | Cancel waiting counts · a Canceler for any thread · end the service (as its drop does). |
Measured through this binding, copied 16-byte messages run at 41 to 100 million per second between two processes, and 64 KiB messages at 435,000 to 665,000 per second (details).