CtrlK
| vimgolf-gym | |
|---|---|
| Overview | |
| Full name | Vimgolf Gym Environment |
| Abbreviation | vimgolf-gym |
| Description | OpenAI gym like, customizable environment and benchmark for Vimgolf |
| Latest version | 0.1.1 |
| Authors | James Brown |
| Organization | Cybergod AGI Research |
| Technical Details | |
| Type | Vim editing challenge evaluation |
| Modality | Text editing, Vim commands |
| Task format | Vim golf challenges
|
| Evaluation metric | Keystroke count (lower is better), Relative inverse score |
| Domains | Text editing, Vim proficiency |
| Languages | Python, Rust |
| Performance
| |
| Saturated | False |
| Resources | |
| Website | Official website |
| GitHub | Repository |
| Dataset | Download |
| License | The Unlicense
|
vimgolf-gym is an OpenAI Gym-like customizable environment and benchmark for Vimgolf, designed to evaluate AI agents' ability to solve text editing challenges using Vim commands efficiently.
vimgolf-gym provides a standardized testing framework for Vim editing proficiency through golf-style challenges where participants must transform given input text to match desired output text using the fewest possible keystrokes.
The environment was created to:
vimgolf-gym consists of several key components:
| Component | Description |
|---|---|
| Challenge Environment | Vim instance with specific input/output requirements |
| Local Challenge Dataset | Cached Vimgolf challenges stored at ~/.cache/cybergod-vimgolf-challenges/ |
| Docker Execution | Containerized Vim environment for isolation |
| Evaluation System | Keystroke counting and solution verification |
The system supports three types of challenges:
| Type | Format | Description |
|---|---|---|
| Test Challenge | "vimgolf-test" | Basic "hello world" challenge |
| Local Challenge | "vimgolf-local-<challenge_id>" | Challenges from local dataset |
| Online Challenge | "vimgolf-online-<challenge_id>" | Challenges from vimgolf.com |
| Custom Challenge | "vimgolf-custom" | User-defined challenges |
The environment provides a Python API for interaction:
```python import vimgolf_gym
env = vimgolf_gym.make("vimgolf-test")
env.act("ihello world\n")
buffer = env.buffer
success = env.verify_keys("ihello world<NL>hello world<Esc>:wq<NL>")
relative_score = env.calculate_relative_inverse_score(score=100) ```
Custom challenges can be defined using YAML format:
```yaml input: |
The second line The first line
output: |
The first line The second line
name: Swap lines description: Swap the first and second lines of the input solution: null ```
| Metric | Description | Formula |
|---|---|---|
| Keystroke Score | Raw keystroke count | Lower is better |
| Relative Inverse Score | Performance relative to worst solution | estimated_worst_solution_score / agent_score |
| Success Rate | Binary completion check | Verified output match |
vimgolf-gym can be installed via multiple methods:
```bash
pip install vimgolf-gym
pip install git+https://github.com/james4ever0/vimgolf-gym.git
docker pull agile4im/cybergod_vimgolf_gym ```
| Screenshot capture and rendering
vimgolf-gym is released under The Unlicense, a public domain equivalent license.