Public Dataset

What people want to use AI for, and which models work

What's in the dataset

Every time someone reaches a Bearing recommendation, we record the structured task classification, the priorities and factor weights the recommender actually applied, which models were recommended, any multi-stage plan, which model the person selected (if any), and -- optionally -- whether it worked. The current schema also records whether recommended/chosen models are open-weight or local-capable, and can capture the coarse hardware/fit context behind a local-model choice. Tasks are included even when no model was selected.

We also publish head-to-head comparison data and Bearing-hosted Route / Trio / Challenger runs. Predicted hardware fit is kept separate from observed execution evidence, so future Ollama or other local-runtime measurements can record what actually ran, on what broad class of hardware, without pretending an estimate was a completed run.

What we never collect: no raw task descriptions, no prompt or response text, no email addresses, no IP addresses, and no browser user-agent or detailed GPU model strings. If someone uses the hardware-aware local-model feature, only a coarse profile such as platform, architecture, memory amount and GPU vendor can be attached to their model choice.

Why this matters

Bearing is building a public record of real-world “task → recommendation → choice → execution → outcome” decisions. Benchmarks test raw capability; Bearing tests fit -- whether a model is the right choice for what someone actually wants to do, including openness and local-execution constraints.

This data is useful for anyone building routing systems, recommendation engines, or evaluation tools for AI models.

Download

Recommendation data

Task classifications, model recommendations, selections, and outcomes.

Download JSON Download CSV

Comparison data

Head-to-head model preferences with task context.

Download JSON Download CSV

Routed-run data

Route, Trio and Challenger runs with recommendation rank, open/local model metadata, blind-judge verdicts and human preferences.

Download JSON Download CSV

Methodology

Task classification: Claude Haiku with a confidence threshold of 0.6. Tasks below this threshold go through a clarification step.
Scoring: 7-factor weighted scoring based on user-ranked priorities. See About for details.
Selection signal: which model the user chose and at what rank, plus a snapshot of whether it was open-weight/local-capable and which open/local/ hardware-fit filters were active.
Execution signal: actual runtime observations are stored separately from predicted hardware fit. The schema can capture runtime, quant, context, coarse hardware, VRAM, latency and tokens/sec when that evidence genuinely exists.
Outcome signal: optional thumbs up/down with structured failure reasons (e.g. quality, speed, cost, hallucination).

Schema

Recommendation dataset

Field	Type	Description
task_type	string	Primary task category
task_subtype	string	Specific task sub-category
complexity	string	low \| medium \| high
input_length	string	short \| medium \| long \| very_long
needs_vision	boolean	Requires image/vision capabilities
needs_tools	boolean	Requires tool use / function calling
needs_code	boolean	Requires code generation
needs_reasoning	boolean	Requires multi-step reasoning
is_recurring	boolean	Recurring or repeated task
data_sensitivity	string	Privacy / on-prem sensitivity class
latency_target	string	realtime \| interactive \| batch
volume	string	Expected usage volume
needs_long_context	boolean	Requires long context
needs_multilingual	boolean	Requires multilingual capability
is_agentic	boolean	Agentic / multi-step tool-using workload
output_length	string	short \| medium \| long \| very_long
mode	string	recommend \| embedding \| pipeline \| validate
priority_order	string[]	User-ranked priority factors
excluded_factors	string[]	Factors the user opted out of
factor_weights	object?	Normalised per-factor weights actually applied
pipeline_stages	object[]?	Multi-stage plan if recommended
classification_schema_version	string	v0.7 \| v0.8 \| v0.9 — task_type enum used
models_recommended	object[]	{slug, rank, weighted_score, model_class, open_weights, is_open_weight, local_capable}
local_recommendations	object[]	Reviewed local candidates with quant, memory footprint and hardware tier
model_selected	object?	Chosen model + selection-time openness/local snapshot + privacy-safe choice context
execution_observations	object[]	Observed runtime/local execution evidence; never inferred from predicted fit
outcome_success	boolean?	User-reported success
failure_reason	string?	Failure reason if applicable
task_date	date	Date the task was created

Comparison dataset

Field	Type	Description
task_type	string	Primary task category
classification_schema_version	string	v0.7 \| v0.8 \| v0.9 — task_type enum used
model_a_slug	string	First model in comparison
model_a_metadata	object	Current openness/local/model-class metadata
model_b_slug	string	Second model in comparison
model_b_metadata	object	Current openness/local/model-class metadata
preferred	string	model_a \| model_b \| tie
preference_reason	string?	Reason for preference
task_date	date	Date of comparison

Routed-run dataset

Field	Type	Description
mode	string	route \| trio \| challenger
task_type	string	Underlying task category
candidates	object[]	Run candidates with rank, score, role and open/local metadata
execution_location	string	bearing_hosted for this dataset
judged_winner	string?	Blind judge choice
human_preferred	string?	Human preferred model or tie
run_date	date	Date of run

Licence

This dataset is released under Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). You are free to share and adapt the data for non-commercial purposes with attribution.

Built by The Good Ship · good-ship.co.uk