An event-loop–based, concurrency-safe matching engine for high-volume transactions and intent pairing
Matchloop is a high-performance matching core designed to process large volumes of concurrent intents (trades, transactions, tasks, bids, requests) using a single-owner event loop, inspired by systems like Nginx, Redis, and high-frequency trading engines.
Instead of scaling by adding threads and locks, Matchloop achieves correctness and efficiency by owning all mutable state in one event loop and scaling horizontally via sharding.
Many systems need to pair or match entities under heavy concurrency, for example:
- Trade / order matching
- Task or job assignment
- Bid–ask pairing
- Ride or delivery matching
- Multiplayer or session pairing
A common implementation pattern is:
mutex.Lock()
queue = append(queue, intent)
mutex.Unlock()This approach:
- Breaks under contention
- Introduces race conditions
- Becomes hard to reason about
- Scales poorly as complexity grows
Matchloop avoids this entirely by adopting a single-writer, event-driven architecture.
State must have exactly one owner.
All mutable state in Matchloop is owned by one event loop goroutine. External callers never touch internal state directly — they send events.
This design:
- Eliminates data races by construction
- Avoids lock contention
- Makes behavior deterministic
- Simplifies reasoning and testing
This is the same philosophy used by:
- Redis (single-threaded core)
- Nginx (event loop)
- Kafka (partition ownership)
- High-performance matching engines
- Event-loop–driven matching engine
- Concurrency-safe by design (no exposed shared state)
- Deterministic matching behavior
- Bounded memory usage
- Backpressure-aware ingestion
- Pluggable matching algorithms
- Graceful shutdown and draining
- Cancellation and timeout handling
Matchloop operates on intents, not domain-specific objects.
This allows it to be used for:
- Trades
- Tasks
- Jobs
- Requests
- Game sessions
- Any high-volume matching problem
type Intent struct {
ID string
Type IntentType
Key string // market, room, topic, symbol, etc.
Priority int // price, skill, weight, etc.
Timestamp time.Time
}Matchloop is a core engine, not a platform.
It intentionally does NOT include:
- Networking / HTTP APIs
- Persistence or durability
- Distributed coordination
- Horizontal auto-scaling
- Redis / Kafka / databases
- UI or dashboards
- Authentication or authorization
These concerns belong outside the matching core.
Clients
↓
Join / Cancel calls
↓
Event channel
↓
┌─────────────────────────────┐
│ Single Event Loop Core │
│ │
│ - Owns all state │
│ - Processes events │
│ - Runs matching algorithms │
│ - Handles timeouts │
└─────────────────────────────┘
↓
Matches channel
There is exactly one goroutine that mutates internal state.
All changes happen via events:
- AddIntent
- CancelIntent
- Timeout
- Shutdown
This enables:
- Deterministic behavior
- Easier testing
- Clear lifecycle management
Matchloop guarantees the following:
- Single ownership of state — no concurrent mutation
- No intent is matched more than once
- No intent is silently dropped
- Each intent is either matched, cancelled, or expired
- FIFO fairness within matching priority
- Bounded resource usage
- No goroutine leaks
If a guarantee cannot be met, it is explicitly documented.
Matching logic is pluggable.
The engine is agnostic to how matching occurs.
type Matcher interface {
Match(book *Book) []Match
}Initial implementations focus on:
- FIFO matching
- Priority-based matching
Future algorithms may include:
- Pro-rata matching
- Time-weighted fairness
- Best-effort batching
Matchloop is explicitly backpressure-aware.
When ingestion capacity is exceeded:
- New intents can be rejected
- Callers are notified immediately
- Internal queues remain bounded
This prevents:
- Unbounded memory growth
- Latency collapse
- Cascading failures
Matchloop scales by sharding, not threads.
Typical strategies:
- One engine per market / symbol
- One engine per partition key
- Multiple engines behind a router
This mirrors:
- Kafka partitions
- Redis sharding
- Exchange matching engines
engine := Matchloop.NewEngine(Matchloop.Config{
GroupSize: 2,
MaxWait: 5 * time.Second,
})
ticket := engine.Join(Matchloop.Intent{
ID: "txn-123",
Type: Matchloop.Buy,
Key: "BTC-USD",
Priority: 1200,
})
select {
case match := <-engine.Matches():
fmt.Println("matched:", match)
case <-time.After(2 * time.Second):
ticket.Cancel()
}On shutdown:
- New intents are rejected
- Pending intents are drained or cancelled
- No goroutines are leaked
- Shutdown is context-aware
Matchloop is not a good fit if:
- You need durable persistence
- You need exactly-once guarantees
- You need distributed consensus
- You need multi-node coordination
- You want a full trading platform
In these cases, use:
- Kafka / Pulsar
- Redis streams
- Dedicated matching systems
Detailed analysis: See PERFORMANCE_ANALYSIS.md
| Metric | Value | Notes |
|---|---|---|
| Submission Throughput | 2.5M intents/sec | Non-blocking channel sends |
| Matching Latency (P50) | 150-180 µs | Pure matching logic |
| Matching Latency (P99) | 274 µs | Under GC pressure |
| End-to-End P50 | 3.9-4.8 ms | Includes queue wait |
| End-to-End P99 | 6.5-8.3 ms | Steady state |
| Tail Behavior (P99.9/P99) | 1.0-1.05 | Minimal outliers |
| GC Pressure | 9 cycles/5s @ 4K/sec | High allocation rate |
- ✅ Core matching is fast: Sub-millisecond (274 µs P99)
⚠️ Queue latency dominates: 3-5 ms waiting in event buffer- ✅ Excellent tail behavior: P99.9/P99 ratio near 1.0
⚠️ Sequential processing: No per-book isolation (single event loop)⚠️ High allocations: ~1.25 KB per intent
- Batch matching systems (5-10 ms latency acceptable)
- Moderate throughput (< 10K intents/sec)
- Predictable, steady load
- Applications where simplicity > ultra-low latency
- Ultra-low-latency HFT (need <1 ms P99 end-to-end)
- Bursty traffic with strict per-book isolation
- Systems requiring hard per-book SLAs
See PERFORMANCE_ANALYSIS.md for detailed optimization roadmap including:
- Object pooling (quick win: 50% GC reduction)
- Batch event processing (1-2 ms queue latency)
- Book sharding (per-book isolation)
🚧 Early development
Current focus:
- Core event loop
- Correctness invariants
- Deterministic matching
Future work (not guaranteed):
- Additional matching strategies
- Metrics hooks
- Shard routing helpers
A live web simulator lets you visualise the engine processing thousands to millions of trades in real time — with throughput charts, latency histograms, a live order book, and per-shard distribution bars.
go run ./cmd/simulator
# Open http://localhost:8080Pick a preset scenario (Baseline · Stress · Peak Load · Ultra) or dial in custom parameters: intent count, submission rate, engine count, buy/sell ratio, cancellation rate, and more. All metrics are streamed live from the actual matchloop engine running inside the server.
- Redis single-threaded architecture
- Nginx event loop model
- Kafka partition ownership
- High-frequency trading matching engines
- Go's CSP concurrency model
MIT
Correctness first. Performance follows. Small cores scale better than complex systems