Merge pull request #18 from SolidLabResearch/codex/repo-hardening

repo: harden runtime and simplify repository layout

Author: argahsuknesib

.github/workflows/fast-pr.yml

@@ -48,8 +48,8 @@ jobs:
       - name: Run Clippy
         run: cargo clippy --all-targets --all-features -- -D warnings
 
-  test:
-    name: Test Suite
+  smoke:
+    name: Integration Smoke
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
@@ -61,8 +61,14 @@ jobs:
       - name: Cache Rust build artifacts
         uses: Swatinem/rust-cache@v2
 
-      - name: Run tests
-        run: cargo test --all-features --verbose
+      - name: Run Janus API integration smoke test
+        run: cargo test --test janus_api_integration_test --all-features
 
-      - name: Run doc tests
-        run: cargo test --doc --all-features --verbose
+      - name: Run HTTP server integration smoke test
+        run: cargo test --test http_server_integration_test --all-features
+
+      - name: Run stream bus CLI smoke test
+        run: cargo test --test stream_bus_cli_test --all-features
+
+      - name: Build HTTP client example
+        run: cargo build --example http_client_example --all-features

GETTING_STARTED.md

@@ -1,134 +1,86 @@
 # Getting Started with Janus
 
-This guide reflects the current state of the repository.
-
-Janus is primarily a backend Rust project. The most useful entry points today are:
-
-- `http_server` for the HTTP/WebSocket API
-- `stream_bus_cli` for replaying RDF files into storage and MQTT
-- the Rust test suite for validating the engine locally
+Janus is a Rust engine for querying historical and live RDF data through one
+Janus-QL model and one HTTP/WebSocket API.
 
 ## Prerequisites
 
-- Rust stable
-- Cargo
-- Git
-- Docker, if you want to run the local MQTT broker
-
-## Clone and Build
-
-```bash
-git clone https://github.com/SolidLabResearch/janus.git
-cd janus
-
-cargo build
-```
+- Rust stable with Cargo
+- Docker and Docker Compose if you want the MQTT-backed replay flow
 
-## Run the Backend
+## Fastest Working Path
 
-### Option 1: Start the HTTP API
+### 1. Build and test
 
 ```bash
-cargo run --bin http_server
+make build
+make test
 ```
 
-The server listens on `http://127.0.0.1:8080` by default.
-
-### Option 2: Inspect the replay CLI
+### 2. Start the HTTP server
 
 ```bash
-cargo run --bin stream_bus_cli -- --help
+cargo run --bin http_server -- --host 127.0.0.1 --port 8080 --storage-dir ./data/storage
 ```
 
-Typical usage:
+Verify it is up:
 
 ```bash
-cargo run --bin stream_bus_cli -- \
-  --input data/sensors.nq \
-  --broker none \
-  --rate 0
+curl http://127.0.0.1:8080/health
 ```
 
-## Run Tests
-
-```bash
-cargo test --all-features
-```
+### 3. Exercise the API
 
-## Run Lint Checks
+The quickest end-to-end client is the example binary:
 
 ```bash
-cargo clippy --all-targets --all-features -- -D warnings
+cargo run --example http_client_example
 ```
 
-## Quick HTTP Flow
+That example covers query registration, start, stop, replay control, and
+WebSocket result consumption.
 
-1. Start the server:
+## Optional Local Demo UI
 
-```bash
-cargo run --bin http_server
-```
+This repository keeps a small static demo at
+`examples/demo_dashboard.html` for manual browser testing.
 
-2. Register a query:
+The maintained Svelte dashboard lives in the separate
+`SolidLabResearch/janus-dashboard` repository.
 
-```bash
-curl -X POST http://localhost:8080/api/queries \
-  -H "Content-Type: application/json" \
-  -d '{
-    "query_id": "demo_query",
-    "janusql": "PREFIX ex: <http://example.org/> SELECT ?s ?p ?o FROM NAMED WINDOW ex:w ON STREAM ex:sensorStream [START 0 END 9999999999999] WHERE { WINDOW ex:w { ?s ?p ?o . } }"
-  }'
-```
-
-3. Start the query:
-
-```bash
-curl -X POST http://localhost:8080/api/queries/demo_query/start
-```
+## Main Binaries
 
-4. Subscribe to results:
+- `http_server`: REST and WebSocket API for query lifecycle and replay control
+- `stream_bus_cli`: replay and ingestion CLI for RDF event files
 
-```text
-ws://localhost:8080/api/queries/demo_query/results
-```
-
-5. Stop the query when done:
+## Common Commands
 
 ```bash
-curl -X POST http://localhost:8080/api/queries/demo_query/stop
-```
-
-## Project Layout
-
-```text
-janus/
-├── src/
-│   ├── api/           # Janus API coordination layer
-│   ├── core/          # RDF event types and encoding
-│   ├── execution/     # Historical execution and result conversion
-│   ├── http/          # HTTP and WebSocket server
-│   ├── parsing/       # JanusQL parser
-│   ├── querying/      # SPARQL execution adapters
-│   ├── storage/       # Segmented storage and indexing
-│   ├── stream/        # Live stream processing
-│   └── stream_bus/    # Replay and broker integration
-├── tests/             # Integration and module tests
-├── examples/          # Example clients and benchmarks
-├── docs/              # Documentation
-└── janus-dashboard/   # Lightweight local demo dashboard
+make build
+make release
+make test
+make fmt
+make fmt-check
+make lint
+make check
+make ci-check
 ```
 
-## Dashboard Boundary
-
-This repository includes a small demo dashboard under `janus-dashboard/`, but the maintained dashboard lives separately:
-
-- `https://github.com/SolidLabResearch/janus-dashboard`
+## Repository Layout
 
-If you are working on frontend product features, use the separate dashboard repository.
+- `src/api`: query lifecycle orchestration
+- `src/http`: REST and WebSocket server
+- `src/parsing`: Janus-QL parsing
+- `src/execution`: historical execution
+- `src/stream`: live stream processing
+- `src/storage`: segmented RDF storage
+- `src/bin`: executable binaries
+- `examples`: runnable examples and a minimal static demo
+- `tests`: integration coverage
+- `docs`: current docs plus older design notes
 
-## Recommended Next Reads
+## Where to Read Next
 
-- [START_HERE.md](./START_HERE.md)
-- [docs/README_HTTP_API.md](./docs/README_HTTP_API.md)
-- [docs/STREAM_BUS_CLI.md](./docs/STREAM_BUS_CLI.md)
-- [docs/README.md](./docs/README.md)
+- `README.md`
+- `START_HERE.md`
+- `docs/DOCUMENTATION_INDEX.md`

README.md

@@ -124,6 +124,15 @@ This example demonstrates:
 - replay control
 - WebSocket result consumption
 
+### Frontend Boundary
+
+The maintained web dashboard lives in the separate
+`SolidLabResearch/janus-dashboard` repository.
+
+This repository keeps a small static demo at
+[`examples/demo_dashboard.html`](./examples/demo_dashboard.html) for manual API
+testing, but frontend development should happen in the dedicated dashboard repo.
+
 ## Development
 
 ### Common Commands
@@ -145,6 +154,8 @@ make ci-check      # local CI script
 The repository includes runnable examples under [`examples/`](./examples), including:
 
 - [`examples/http_client_example.rs`](./examples/http_client_example.rs)
+- [`examples/comparator_demo.rs`](./examples/comparator_demo.rs)
+- [`examples/demo_dashboard.html`](./examples/demo_dashboard.html) for a minimal local demo
 
 ## Documentation
 

START_HERE.md

@@ -1,54 +1,32 @@
-# Janus Backend - Start Here
-
-Use this file if you want the fastest path to a working local backend.
+# Janus Start Here
 
 ## Quick Start
 
-### 1. Start the MQTT broker
-
 ```bash
 docker-compose up -d mosquitto
+cargo run --bin http_server -- --host 127.0.0.1 --port 8080 --storage-dir ./data/storage
+curl http://127.0.0.1:8080/health
 ```
 
-### 2. Start the HTTP server
-
-```bash
-cargo run --bin http_server
-```
-
-### 3. Check health
-
-```bash
-curl http://localhost:8080/health
-```
-
-Expected response:
-
-```json
-{"message":"Janus HTTP API is running"}
-```
-
-## Optional: Open the local demo dashboard
-
-This repository contains a small demo dashboard:
+In another terminal, run:
 
 ```bash
-open examples/demo_dashboard.html
+cargo run --example http_client_example
 ```
 
-For the maintained frontend, use the separate repository:
-
-- `https://github.com/SolidLabResearch/janus-dashboard`
-
-## Most Useful Docs
+## What To Use
 
-- [GETTING_STARTED.md](./GETTING_STARTED.md)
-- [docs/README_HTTP_API.md](./docs/README_HTTP_API.md)
-- [docs/QUICKSTART_HTTP_API.md](./docs/QUICKSTART_HTTP_API.md)
-- [docs/README.md](./docs/README.md)
+- `http_server` is the main backend entry point
+- `stream_bus_cli` is the ingestion and replay CLI
+- `examples/demo_dashboard.html` is a minimal manual demo
+- the maintained Svelte dashboard lives in the separate `janus-dashboard` repository
 
-## Notes
+## Current Docs
 
-- The backend is the primary concern of this repository.
-- `http_server` is the main user-facing executable.
-- `stream_bus_cli` is the replay/ingestion CLI.
+- `README.md`
+- `GETTING_STARTED.md`
+- `docs/DOCUMENTATION_INDEX.md`
+- `docs/HTTP_API_CURRENT.md`
+- `docs/README_HTTP_API.md`
+- `docs/QUICKSTART_HTTP_API.md`
+- `docs/QUICK_REFERENCE.md`

docs/DOCUMENTATION_INDEX.md

@@ -12,6 +12,7 @@ This is the shortest path to understanding the current Janus implementation.
 6. [BASELINES.md](./BASELINES.md)
 7. [HTTP_API_CURRENT.md](./HTTP_API_CURRENT.md)
 8. [ANOMALY_DETECTION.md](./ANOMALY_DETECTION.md)
+9. [QUICK_REFERENCE.md](./QUICK_REFERENCE.md)
 
 ## What Each File Covers
 
@@ -38,29 +39,33 @@ This is the shortest path to understanding the current Janus implementation.
   - current REST endpoints
   - WebSocket result flow
   - request and response shapes
-  - `baseline_mode` registration fallback
+  - persisted query lifecycle status
 
 - [ANOMALY_DETECTION.md](./ANOMALY_DETECTION.md)
   - when extension functions are enough
   - when baseline state helps
   - recommended query patterns
 
+- [QUICK_REFERENCE.md](./QUICK_REFERENCE.md)
+  - common local commands
+  - query lifecycle endpoints
+  - replay endpoints
+  - smoke-test flow
+
 ## Additional Current Guides
 
 - [STREAM_BUS_CLI.md](./STREAM_BUS_CLI.md)
 - [README_HTTP_API.md](./README_HTTP_API.md)
 - [QUICKSTART_HTTP_API.md](./QUICKSTART_HTTP_API.md)
-- [QUICK_REFERENCE.md](./QUICK_REFERENCE.md)
 
 ## Legacy Material
 
 The following files remain useful as background, but they are not the main entrypoint for the current code:
 
 - [ARCHITECTURE.md](./ARCHITECTURE.md)
 - [EXECUTION_ARCHITECTURE.md](./EXECUTION_ARCHITECTURE.md)
-- [HTTP_API.md](./HTTP_API.md)
-- [MVP_TODO.md](./MVP_TODO.md)
 - [MVP_ARCHITECTURE.md](./MVP_ARCHITECTURE.md)
+- [MVP_TODO.md](./MVP_TODO.md)
 - [RSP_INTEGRATION_COMPLETE.md](./RSP_INTEGRATION_COMPLETE.md)
 - [SPARQL_BINDINGS_UPGRADE.md](./SPARQL_BINDINGS_UPGRADE.md)