Backprop Brings W&B-Style Tracking to Local SQLite | Zeniteq
Loading
Loading article
Backprop Brings W&B-Style Tracking to Local SQLite
Backprop 0.1.3 offers a familiar Python API, local dashboards and artifact storage, with Google Cloud and Palantir Foundry backends for other deployment needs.
Backprop 0.1.3 provides machine learning teams with a local experiment tracking alternative that mimics the Weights & Biases API without requiring a hosted account. The project enables local dashboarding and artifact storage by utilizing a Python SDK, a browser-based UI, and a SQLite backend.
The release supports local SQLite, Google Cloud, and Palantir Foundry backends. Users can run the server via the backprop serve command on port 8765, while a compatibility launcher allows existing scripts to redirect wandb imports. The documentation notes specific constraints, including a 512 MiB file size limit and a requirement for Python 3.10 or newer.
Backprop 0.1.3 gives machine-learning teams a documented route to local experiment tracking without a hosted Weights & Biases account. It combines a Python SDK, a browser dashboard and three backend options: local SQLite, Google Cloud and Palantir Foundry. A compatibility launcher can redirect existing wandb imports inside a training process, reducing the initial code changes needed to try it. The project README describes the supported setup.
Two boundaries matter before planning a migration. Backprop implements compatibility at the Python API level, so pointing the official W&B SDK at a different server address will not work. Its SQLite server also accepts only loopback connections, making it a single-machine service with SSH forwarding for remote access.
Backprop is a young project with limited documented adoption. Its repository provides no clear launch timestamp, and the operational behavior described here comes from project documentation, without independent testing. I’d treat it as a candidate for a contained pilot before making it the system of record for important training runs.
What Backprop Stores and Displays
Backprop’s dashboard covers run comparisons, metric charts, configuration, logs, files, tables, media and artifacts. The same public Python APIs and browser views serve all three backends, according to the README.
artifacts/ stores immutable media and artifact contents.
connection.json tells clients which local installation to use.
SDK journals retain retryable events and staged files for individual training runs.
That separation affects backup planning: copying the SQLite database alone leaves artifact bytes behind.
The local backend documentation says the SDK writes a durable journal and uploads batches every five seconds by default. The server commits events and offsets in SQLite transactions, while file bytes reach durable storage before their database references commit.
The browser refreshes its event view every 30 seconds. These documented intervals mean the dashboard should be understood as periodically refreshed monitoring, with some delay between logging a value and seeing it appear.
Work with Zeniteq
Promote your app with us
Reach readers exploring AI tools and workflows. Discover sponsorship and advertising options for your app.
The first command installs release 0.1.3 from GitHub. The second starts the local dashboard and storage API, using the specified directory for persistent data.
The default address is 127.0.0.1:8765. The installed package includes dashboard assets, so this local setup needs no source checkout, Node.js installation or separate database service. Restarting against the same data directory preserves the installation identity and stored runs, according to the local setup guide.
In a second terminal, point the training environment at the generated connection file:
Local connections require no token or browser sign-in. The local transport ignores existing FOUNDRY_TOKEN and BACKPROP_TOKEN variables, so credentials left in the environment are not sent through it.
Keep Existing Imports With the Launcher
Backprop offers two ways to adopt its Python API.
For code you are willing to edit, the documented pattern aliases Backprop to the familiar name:
import backprop as wandb
with wandb.init(
project="local-experiment",
config={"lr": 1e-4},
) as run:
for step in range(100):
run.log({"train/loss": 1 / (step + 1)}, step=step)
run.summary["result"] = "complete"
print(run.url)
This example creates a run, records configuration, logs a loss value at each step and adds a summary field. The printed URL leads to the run’s dashboard view.
For a script that already contains import wandb, the documented alternative is:
python -m backprop run -- training.py
A module-based entry point is also supported:
python -m backprop run -- -m your_training_module
The launcher redirects W&B imports within that Python process. It provides an incremental adoption path, but the README explicitly says WANDB_BASE_URL compatibility is unsupported.
That distinction changes the migration checklist. Backprop substitutes a Python implementation; it is not an endpoint for the official W&B client. I’d audit a representative training job’s actual API usage before expanding a pilot, especially where framework integrations or less common logging features are involved.
Optional dependencies need attention too. Images and arrays, video, plotting, dataframes and framework integrations use separate extras. Backprop says it never installs these packages automatically when a feature is invoked. From a source checkout, for example, python -m pip install '.[media,plots]' installs the dependencies for those two capability groups.
Offline Logging and SQLite Are Different Modes
Backprop also documents offline logging without a Foundry account:
import backprop
with backprop.init(project="demo", mode="offline") as run:
run.log({"loss": 0.5})
This example records a run offline. The local SQLite setup adds a running storage API and browser dashboard. Both avoid a hosted account, but they serve different purposes: offline logging captures records without an active backend connection, while the local server makes stored runs available for inspection. The README documents both paths.
For teams training on disconnected machines, that distinction is useful. Capture can happen separately from the later step of importing records into a dashboard-backed installation.
Import Journals Without Assuming Full W&B Migration
Backprop’s local guide documents an importer for existing SDK journals:
The directory name needs careful interpretation. The guide describes Backprop’s journal records, acknowledgement files and uploader. It does not document a general converter for every native W&B run format or a complete export tool for hosted W&B history.
That makes future logging and historical migration separate decisions. The launcher offers a route for new runs; historical imports require compatible journals and accessible files.
The importer preserves original records and acknowledgement state while tracking local ingestion separately. Repeating an import picks up new records, and identical event IDs do not create additional records. Conflicting duplicates or a changed imported prefix fail explicitly; an incomplete final line waits for a later import.
Artifact availability is another boundary. Locally staged files must remain readable until import finishes. The importer does not fetch missing bytes from Foundry, and remote-only artifacts still require their original backend.
My reading is that this is a cautious replay mechanism, useful for preserving source state during a trial. It should not be mistaken for automatic historical portability between tracking services.
Remote Access Uses SSH, Not a Public Bind
The local server refuses public bind addresses. It checks browser origins and provides neither multi-user access control nor a public TLS service. Projects within one installation share its files.
For a server running on a remote training machine, the backend README documents SSH forwarding:
ssh -N -L 8765:127.0.0.1:8765 training-host
This forwards port 8765 on your machine to the server’s loopback port on training-host, allowing a local browser to reach the remote dashboard.
The practical implication is important: “self-hosted” does not make the SQLite backend a ready-made shared team portal. Its documented access model suits a workstation or a trusted remote machine reached through SSH. Teams needing individual permissions and a shared browser URL should evaluate the other backend deployments separately.
Backprop says all three backends share APIs and dashboard components. Foundry setup adds a deployed browser application, a connection file and token-based authentication. Selecting another BACKPROP_CONFIG changes the backend the SDK uses; that configuration switch is not a documented transfer of existing stored history.
Plan for Storage Limits and Backups
The local backend lists several operational limits:
Item
Documented limit or behavior
Event size
900 KB
File size
512 MiB
Browser event limit
One million events
Streams and branches
One stream partition and the master branch
Automatic retention
None
Artifact garbage collection
None
The 512 MiB file cap deserves an early check if a workflow logs large model checkpoints. The lack of automatic retention also means storage management remains the operator’s responsibility. These are documented constraints, not measured performance results.
For backups, the project instructs operators to flush training writers, stop the server and copy the complete data directory, including artifacts and any remaining SQLite journal files. SDK journals and browser-local preferences remain separate.
I’d make artifact download and backup restoration part of the first evaluation. Those checks address the practical reason to own an experiment tracker locally: keeping runs readable and recoverable after the training process has ended.
Frequently Asked Questions
4 questions
1
Does Backprop Need a Hosted W&B Account?
No. The documented local SQLite mode requires no token or browser sign-in. Offline logging also works without a Foundry account.
2
Can I Keep import wandb in Training Code?
The compatibility launcher redirects those imports inside its Python process. Compatibility is API-level; changing WANDB_BASE_URL on the official SDK will not work.
3
Can I Import All My Existing W&B Runs?
The documentation supports importing compatible local SDK journals. It does not establish a general migration path for every native W&B format or an entire hosted account.
4
Can a Team Share the Local Dashboard?
The SQLite server is loopback-only and has no multi-user access controls. SSH forwarding is the documented way to access it on a remote machine.
