OpenResearch

OpenResearch is an open-source workspace for coordinating coding agents during computational research. Developed by alphaXiv, it combines a desktop app, a local web dashboard, a command-line interface named orx, git-based experiment tracking, and adapters for local and remote compute. The software is designed to keep project state on the user's machine while allowing selected jobs and service requests to use external systems.^[1][2][13]

OpenResearch is distinct from arXiv, the alphaXiv paper-reading service, generic "autoresearch" workflows, and products called Deep Research. Its name refers to alphaXiv's research-agent workspace rather than a publication repository or a research model. The repository and released source are licensed under the MIT License.^[3][4] As of October 25, 2026, the latest release is v0.2.21, published on October 23, 2026, and the GitHub repository, now named alphaXiv/OpenResearch, showed about 7,500 stars and 504 forks.^[11][21] In October 2026 alphaXiv built support for Trackio into OpenResearch, so that agents can log metrics, view charts beside each run, and read the logged curves back when choosing the next experiment.^[14][18]

History and releases

alphaXiv created the GitHub repository on June 7, 2026, and published its first tagged release, v0.1.1, the same day.^[21][31] The company announced OpenResearch publicly on X on July 21, 2026, describing it as a way to "run parallel research agents with any model" in which "your code + data stays with you".^[20] The repository was first published as alphaXiv/openresearch-cli. Release notes switched to the alphaXiv/OpenResearch name with v0.1.119 on September 3, 2026, and links that use the old name redirect to the new one.^[31] Releases have been frequent: 142 GitHub releases were published between June 7 and October 23, 2026.^[31]

As of October 25, 2026, the README recommends the desktop app over the standalone CLI. It lists builds for macOS 11 or later, Windows (labelled beta, which requires Git for Windows), and Linux on x86_64 and ARM64 (the Linux app needs glibc 2.35 or later). The standalone CLI install script targets macOS and Linux. The README notes that device-management policies on managed Macs may block the CLI installed by the script, because it is not yet signed, and points those users to the signed and notarized desktop app.^[13]

Workspace and experiment model

Running orx up starts an embedded dashboard and JSON/SSE API on 127.0.0.1, backed by a local SQLite database; the current README gives the dashboard address as http://127.0.0.1:4791.^[1][13] The store records projects, conversations, experiment metadata, runs, logs, and related workspace state. Creating a local project does not publish its repository. An OpenResearch account is used for service-operated features such as organizations and managed compute, not as a prerequisite for the basic local workspace.^[1][2]

Each agent session receives an isolated git worktree. Experiments form a parent-child tree in which the root represents a baseline and child nodes represent variants. The implementation records parent experiment identifiers and git references, while a run is associated with a committed source snapshot. This structure preserves lineage between a proposed change, the code revision that was run, and the resulting logs or artifacts. It does not by itself establish that an experiment is scientifically valid or reproducible on a different environment.^[1][5] Later releases added an orx exp move command for re-parenting experiment subtrees (v0.2.19) and lineage navigation in the experiment table and overview (v0.2.20).^[31][12]

The dashboard can display agent conversations, experiment branches, diffs, files, run status, and logs. The CLI includes commands for listing projects and runs, reading logs, launching experiments, searching literature (orx discover), and fetching papers (orx paper).^[1] These interfaces organize evidence produced during a workflow; they do not independently check whether an agent's interpretation of that evidence is correct.

Experiment tracking with Trackio

Trackio's developers describe it as "a lightweight, local-first, and free experiment tracking library from Hugging Face".^[19] In v0.1.117, the built-in reproduce-paper prompt told agents to prefer Weights & Biases when the user wanted metrics logged.^[30] An issue opened on October 7, 2026 said that this agent guidance was the only tracking OpenResearch knew about, and proposed support for self-hosted Trackio servers and a file-only TensorBoard mode.^[32]

Support arrived in two steps. Version 0.2.20 (October 21, 2026) gave every run an ORX_RUN_ID variable (previously only Tinker runs received one) and told agents, through the orx-compute skill, to name Trackio runs with it.^[15][12] The same release gave local, Tinker, SSH and Slurm runs a per-run TENSORBOARD_LOGDIR directory, added an orx tensorboard <runId> command that prints the command needed to open it, added a Settings check for a saved Trackio server, and made remote launches print a warning when the saved Trackio URL points at localhost.^[16]

Version 0.2.21 (October 23, 2026) included a pull request that, in its author's words, "makes Trackio a first-class part of orx for the Trackio launch".^[14][11] According to that pull request and the current compute skill:^[14][17]

Every run now receives ORX_EXPERIMENT, ORX_EXPERIMENT_TITLE, ORX_PARENT_EXPERIMENT (when it has a parent), ORX_COMMIT and TRACKIO_PROJECT, and the compute skill gives agents a trackio.init call that uses these values as the run name, group and config. Saved environment variables reach every backend except Hugging Face Jobs, which receive only the ORX_* and Trackio variables; the pull request says Jobs get the Trackio URL and write token as secrets.^[14][17] A Visualize tab opens a run's Trackio charts inside the app, and only for runs that actually logged. Private Spaces are embedded with a read-only signed token from Hugging Face, and the pull request says the Hugging Face token is only sent to *.hf.space hosts, and only for private Spaces.^[14]

For agent read-back, orx metrics <runId> summarizes what runs logged to Trackio: for each metric, the first, last, minimum and maximum values with their steps, the mean over the last tenth of the run, a sparkline, and sampled step-value points. The skill tells agents to read it while a run is live and after it ends, and to pass the parent experiment's run as well so that curves, not just final numbers, are compared.^[17] The pull request's test plan listed several paths as not yet tested at merge, including the self-hosted server path after a review refactor, public Space embeds, an agent using orx metrics in chat to compare runs, and the local dashboard on Windows.^[14]

alphaXiv's October 23 announcement summarized the feature as: "Your agent logs its experiments to Trackio, charts open right next to each run, and it reads the curves back to decide what to try next." It added that users running on Hugging Face Jobs, Modal, Kubernetes or Slurm can save a Hugging Face Space so that "every run lands there, private Spaces included".^[18] Weights & Biases remains available: in v0.2.21 the reproduce-paper prompt tells agents to use Trackio for optional tracking and to use Weights & Biases only when the user asks for it.^[35]

Agent and compute integrations

As of v0.2.21, the README says OpenResearch works natively with Claude Code, Codex, OpenCode, Cursor, and Google Antigravity, and that local models can be used with OpenCode through LM Studio, oMLX, Ollama, or a custom endpoint.^[13] Version 0.2.20 added configurable ACP harnesses; the pull request lists detection for adapters including DeepSeek Harness, Kimi Code, MiniMax Code, Devin CLI and Grok Build.^[22] The user selects a harness and model for a session, and the system invokes the corresponding locally installed agent software. The repository includes separate integration code for the supported harnesses rather than providing its own foundation model.^[1][5] A model-picker control added in v0.2.15 switches between "Agentic" and "Copilot" autonomy.^[31]

For literature, the README says OpenResearch integrates with alphaXiv, bioRxiv and PubMed so that agents can find relevant papers.^[13] In the v0.2.21 source, the paper and discovery commands also cover OpenAlex.^[23]

Experiment runs can execute on the local machine, over SSH, or through documented adapters for Slurm, Kubernetes, Ray Jobs, Hugging Face Jobs, Modal, Tinker, and OpenResearch managed compute. The remote backends receive a committed source snapshot or use backend-specific staging. Tinker is a special case: OpenResearch runs a local controller while Tinker's model operations occur remotely, and the user must separately install and configure the Tinker SDK.^[1][6][17] Recent releases added CPU and memory requests for Slurm runs (v0.2.19) and a configurable remote storage root for SSH and Slurm (v0.2.21).^[31][11] The openresearch.sh documentation describes the managed service as "a compute marketplace for research workloads" that compares live GPU and CPU availability across providers and launches instances from the web or the orx CLI.^[28]

The project describes an autonomous loop that can propose a change, modify code, launch an experiment, inspect results, and select a next step. The v0.2.21 tree ships 14 agent skills, including orx-lit-review, orx-paper (drafting a paper as LaTeX), orx-experiment-tree and orx-compute, alongside a built-in reproduce-paper workflow.^[36][35] These are released orchestration capabilities and prompts, not proof that the system can replicate any paper or complete open-ended research reliably.^[1][5]

Local-first scope and external services

The project's "local-first" description applies to its default workspace state: projects, conversations, experiments, run records, code, logs, and artifacts are stored locally. It should not be read as a claim that every byte remains on the device. Agent providers may receive prompts and context under their own configurations. Jobs sent to SSH hosts or cloud backends necessarily transfer source snapshots, commands, data, or outputs required for those runs. Paper search, managed compute, account functions, and optional integrations, including a Trackio Space or server, also contact external services.^{[1][2][6][14]}

As documented for v0.1.117, official release builds send opt-out usage analytics to the OpenResearch API. The documented event fields include a random installation identifier, CLI version, operating system and architecture, a continuous-integration flag, command names, and coarse event labels. The project says these events exclude code, prompts, file contents and paths, repository names, tokens, email addresses, and project or experiment identifiers. Persistent opt-out is available through orx telemetry off, and --no-telemetry disables it for one invocation. Source and development builds do not send production analytics.^[1][7]

The scope widened in later versions. As of October 2026, the README says official builds send coarse analytics events "and detailed agent feedback" in the background, and that anonymized token and message counts are used to identify which models are used across OpenResearch, published at openresearch.sh/rankings.^[13] That page says its data collection began on October 2, 2026, and charts daily token usage by model.^[27] The v0.2.21 telemetry module states that the onboarding research profile includes selected categories and user-entered area or background text.^[24] The README says all usage analytics and agent feedback can be turned off in the desktop app's settings or with orx telemetry off.^[13]

Agent feedback comes from the orx-feedback skill, which tells agents to file bug reports, feature requests or "frustration" reports to the OpenResearch team with orx feedback without interrupting the current task, and to leave out code, prompts, transcripts, logs, credentials and unpublished research details.^[33] On October 14, 2026, a user opened an issue arguing that the skill hid the sending of analytics from user compute, contrary to the tool's local-first marketing. alphaXiv closed it the same day with a pull request that disclosed background agent feedback in the README, clarified that the analytics opt-out also disables it, and changed only README and skill wording.^[25][26]

The implementation contains one narrow exception to the opt-out rule. When a user changes the telemetry choice in an eligible official build, it attempts to send a cli_telemetry_consent event. A declined choice uses a shared sentinel identifier rather than generating a per-install identifier. After opt-out, ordinary product telemetry is disabled.^[7][24]

Security considerations

The dashboard binds to the IPv4 loopback address rather than a public interface, but its application routes do not require authentication. On a single-user computer, the loopback boundary limits ordinary network exposure. It is not an authorization boundary between users or processes that can reach the same host.^[1][2][34]

This limitation was especially relevant to orx up --remote in v0.1.117. That command started the dashboard on a remote host and created an SSH tunnel to the user's local machine, and the README of that version warned that the remote service bound to loopback with no application-level authentication, so other users on that host could reach it.^[1] The v0.2.21 source describes the command differently: it "starts an authenticated server there", and in that persistent host mode API requests must carry a per-session bearer token, while some actions are refused in SSH workspaces. The ordinary local dashboard remains loopback-only without application-level authentication.^[23][34]

OpenResearch also coordinates tools that can edit repositories, execute commands, submit compute jobs, and access configured credentials. Its git worktrees separate concurrent sessions at the filesystem and branch level, but they are not security sandboxes. The safety properties of a run depend on the chosen agent, its permissions, the host, external services, and the compute backend.^[5][6]

Evaluation and limitations

OpenResearch's website and documentation position the product as a way to reproduce papers and run autonomous research loops. Its inclusion as an optional harness in an ICML 2026 agent-reproducibility challenge documents external use, but the challenge FAQ also permits participants to use other agents and does not validate OpenResearch's effectiveness.^[8]

Research on coding agents provides relevant context without evaluating this product. A 2026 social-science reproduction study found large differences between Claude Code and Codex and reported execution failures for some tasks, showing that outcomes can depend strongly on the underlying agent and materials.^[9] Separate work on scientific machine-learning replication argues that a final agent response is insufficient evidence and that completion should be tied to workspace artifacts and validation checks.^[10] RECLAIM, a September 2026 benchmark of 100 NeurIPS 2025 papers that grades runs from logs and outputs rather than agents' own reports, ran four agents once per paper; the best agent in each tier reproduced 41% of papers whose code, data and weights were released, 27% when weights were missing, and 15% when code was missing. The paper does not mention OpenResearch.^[29] OpenResearch's experiment tree, archived run evidence and Trackio curves can support that kind of practice, but they do not replace domain review, statistical checks, environment capture, or independent replication.

The software changed rapidly through the 0.1.x series in August 2026 and the 0.2.x series in September and October 2026. Backend coverage, data handling, telemetry scope, security behavior, and agent integrations therefore require version-specific verification. Claims such as "replicate any paper" or fully autonomous research are best understood as product positioning rather than demonstrated general capabilities.

Continue reading

• CCoding agentA coding agent is a software system that uses a large language model to carry out programming work by acting on a development environment rather than only producing text for a person to copy.Mentioned twice above
• TTrackioTrackio is an open-source experiment tracking library for Python developed by Hugging Face. It records metrics, configurations and media from machine learning training runs and displays them in a dashboard.Mentioned 28 times above
• Hugging FaceHugging Face is an artificial intelligence company and collaboration platform for machine learning.Mentioned 9 times above
• TTinker (Thinking Machines Lab)Tinker is a fine-tuning API for open-weight language models, built by Thinking Machines Lab, the artificial intelligence startup founded by former OpenAI chief technology officer Mira Murati.Mentioned 6 times above
• SSlurmSlurm is an open-source workload manager and job scheduler for Linux clusters, developed and maintained by SchedMD, which NVIDIA acquired in December 2025.Mentioned 5 times above
• opencode (SST)opencode is an open-source AI coding agent built for the terminal. Developed by Anomaly Innovations, formerly known as the SST (Serverless Stack) team, it provides a terminal user interface (TUI) through which…Mentioned twice above

More to read

• Claude Code
• Kubernetes
• Antigravity (Google)
• Ollama
• Ray (framework)
• Cursor (code editor)

Topics:AI AgentsAI ResearchDeveloper ToolsOpen Source AI

References

1. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12alphaXiv, "OpenResearch" repository README
2. ^1 ^2 ^3 ^4OpenResearch, official website
3. ^1 ^2alphaXiv, OpenResearch CLI v0.1.117 release
4. ^alphaXiv, OpenResearch MIT License
5. ^1 ^2 ^3 ^4alphaXiv, OpenResearch v0.1.117 source tree
6. ^1 ^2 ^3alphaXiv, OpenResearch compute skill documentation
7. ^1 ^2alphaXiv, OpenResearch telemetry implementation
8. ^ICML 2026 Agent Reproducibility Challenge, FAQ
9. ^Alizadeh et al., "AI Coding Agents Can Reproduce Social Science Findings"
10. ^"Coding agents can replicate scientific machine learning papers"
11. ^1 ^2 ^3 ^4alphaXiv, OpenResearch v0.2.21 release notes (October 23, 2026)
12. ^1 ^2 ^3alphaXiv, OpenResearch v0.2.20 release notes (October 21, 2026)
13. ^1 ^2 ^3 ^4 ^5 ^6 ^7alphaXiv, OpenResearch README at v0.2.21
14. ^1 ^2 ^3 ^4 ^5 ^6 ^7alphaXiv/OpenResearch pull request #756, "Trackio: Spaces, hf runs, zero-setup local, Visualize view, and orx metrics"
15. ^alphaXiv/OpenResearch pull request #679, "Give every run ORX_RUN_ID and point agents at Trackio"
16. ^alphaXiv/OpenResearch pull request #680, "Add TensorBoard log dirs, orx tensorboard, and Trackio checks and links"
17. ^1 ^2 ^3 ^4alphaXiv, orx-compute skill at v0.2.21
18. ^1 ^2alphaXiv (@askalphaxiv), post announcing Trackio in OpenResearch, October 23, 2026
19. ^gradio-app/trackio GitHub repository
20. ^alphaXiv (@askalphaxiv), "Introducing OpenResearch", July 21, 2026
21. ^1 ^2alphaXiv/OpenResearch GitHub repository
22. ^alphaXiv/OpenResearch pull request #510, "Add configurable ACP harnesses (OR-378)"
23. ^1 ^2alphaXiv, OpenResearch CLI definitions (src/main.rs) at v0.2.21
24. ^1 ^2alphaXiv, OpenResearch telemetry implementation at v0.2.21
25. ^alphaXiv/OpenResearch issue #658, "OpenResearch Skill Harness Intentionally hides the sending of analytics using user compute"
26. ^alphaXiv/OpenResearch pull request #662, "docs: clarify background feedback and research exclusions"
27. ^OpenResearch, "AI model rankings for autoresearch"
28. ^OpenResearch, "Getting started with OpenResearch" documentation
29. ^Salunkhe et al., "RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?"
30. ^alphaXiv, OpenResearch built-in slash skills (src/local/skills.rs) at v0.1.117
31. ^1 ^2 ^3 ^4 ^5 ^6 ^7 ^8 ^9 ^10 ^11 ^12alphaXiv, OpenResearch GitHub releases
32. ^alphaXiv/OpenResearch issue #571, "Support Trackio, and file-only TensorBoard, for experiment tracking"
33. ^alphaXiv, orx-feedback skill at v0.2.21
34. ^1 ^2alphaXiv, OpenResearch dashboard server (src/commands/up.rs) at v0.2.21
35. ^1 ^2alphaXiv, OpenResearch built-in slash skills (src/local/skills.rs) at v0.2.21
36. ^alphaXiv, OpenResearch agent skills at v0.2.21

Show all 36 references