Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 0 additions & 9 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,6 @@
DATABASE_URL=sqlite:///./techapi.db
DATABASE_POOL_SIZE=10

# === Supabase (production Postgres host) ===
SUPABASE_URL=https://xxx.supabase.co
SUPABASE_ANON_KEY=eyJ...
SUPABASE_SERVICE_KEY=eyJ... # secret, never expose to clients

# === API ===
API_HOST=0.0.0.0
API_PORT=8000
Expand All @@ -22,10 +17,6 @@ CORS_ORIGINS=http://localhost:3000,https://techpicks.app

# === Logging ===
LOG_LEVEL=INFO
SENTRY_DSN=https://... # optional

# === External Services ===
REDIS_URL=redis://localhost:6379 # optional (Phase 2+)

# === Scoring ===
SCORING_CONFIG_PATH=./config/scoring.yaml
6 changes: 2 additions & 4 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
# Default owners for everything in the repo.
* @GetTechAPI/maintainers

/app/ @GetTechAPI/maintainers
# Default owner for everything in the repo.
* @Seungpyo1007
9 changes: 5 additions & 4 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: "🐛 Bug report"
description: Report a bug in the API or tooling
description: Report a bug in the validator, API server, dump generator, collectors or verification
labels: ["type:bug"]
body:
- type: textarea
Expand All @@ -13,9 +13,10 @@ body:
id: reproduce
attributes:
label: Steps to reproduce
description: Include the exact request (curl) where possible.
description: Include the exact command or request, and the TechEngine commit you ran.
placeholder: |
curl https://api.techapi.dev/v1/smartphones/galaxy-s25
python -m app.validate
curl http://localhost:8000/v1/smartphones/galaxy-s25
validations:
required: true
- type: textarea
Expand All @@ -28,4 +29,4 @@ body:
id: request-id
attributes:
label: request_id (if any)
description: The `request_id` from the error response helps us trace it.
description: For API errors, the `request_id` from the error response.
8 changes: 8 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Missing or incorrect data
url: https://github.com/GetTechAPI/TechAPI/issues/new/choose
about: This repository holds no data. Report a missing record or a wrong value in the TechAPI repository.
- name: Video games
url: https://github.com/GetTechAPI/game-catalog/issues
about: Game records live in the game-catalog repository.
29 changes: 0 additions & 29 deletions .github/ISSUE_TEMPLATE/data_addition.yml

This file was deleted.

28 changes: 0 additions & 28 deletions .github/ISSUE_TEMPLATE/data_correction.yml

This file was deleted.

8 changes: 1 addition & 7 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,12 @@ env/
# TECHAPI_DATA_DIR); the dataset is never committed here.
/TechAPI/

# Generated static dump (build artifact; published by refresh-data workflow)
# Generated static dump (build artifact; deploy-pages.yml publishes it)
/dump/

# Local-only handoff notes (private; never committed to the public repo)
*.local.md

# Note: data/_staging/ (raw collected candidate pool) is intentionally tracked —
# comprehensive data collection is a purpose of this repo.

# Local Wikipedia GPU backfill cache and dry-run reports (never a TechAPI write)
.wikipedia-gpu-backfill/

Expand All @@ -55,6 +52,3 @@ coverage.xml

# Local Wikipedia SoC backfill cache and reports (never commit)
.wikipedia-soc-backfill/

# Recomputable verifier cache
/data/_verify/state/
7 changes: 4 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,10 @@ COPY pyproject.toml README.md ./
COPY app ./app
RUN pip install --no-cache-dir .

# Runtime assets.
COPY data ./data
COPY scripts ./scripts
# Runtime assets. The dataset is not part of the image: mount a TechAPI
# checkout's data/ at /data (see docker-compose.yml).
COPY config ./config
ENV TECHAPI_DATA_DIR=/data

# Run as a non-root user.
RUN useradd --create-home --uid 10001 appuser \
Expand Down
20 changes: 14 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ tests/ # unit/, integration/, verify/
site/ # Astro landing page, deployed to GitHub Pages
docs/ # SPEC, DATA_PIPELINE, DEVELOPMENT
.github/workflows/ # CI; every workflow is listed below
Dockerfile, docker-compose.yml # stale, do not build (see Docker)
Dockerfile, docker-compose.yml # API + Postgres, dataset mounted (see Docker)
```

Routes under `/v1`: `health`, `version`, `brands`, `socs`, `smartphones`,
Expand Down Expand Up @@ -115,11 +115,19 @@ Smartphones continue to require numeric values for both fields.

### Docker

`Dockerfile` and `docker-compose.yml` predate the split from TechAPI and do
not work as they are: the Dockerfile copies `data/` and `scripts/`, which no
longer exist here, so the image does not build, and the compose file mounts no
TechAPI checkout for `app.seed` to read. No workflow builds or uses them. Use
the Quickstart above; for Postgres, point `DATABASE_URL` at your own instance.
The image holds the engine only and expects the dataset at `/data`
(`TECHAPI_DATA_DIR`). `docker-compose.yml` starts Postgres and the API, and
mounts `../TechAPI/data` read-only at that path, so clone the two repos side by
side:

```text
<parent>/TechEngine # this repo; run `docker compose up --build` here
<parent>/TechAPI # the dataset
```

The API seeds the database from the mount before it serves, which takes a
while on the full dataset. The image has no git, so record timestamps are the
seed time. No workflow builds or uses these files.

## Roadmap

Expand Down
6 changes: 6 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ services:
API_ENV: development
ports:
- "8000:8000"
# The dataset comes from a TechAPI checkout next to this repo:
# <parent>/TechEngine (this repo; run docker compose here)
# <parent>/TechAPI (git clone https://github.com/GetTechAPI/TechAPI.git)
# Its data/ is mounted read-only where the image's TECHAPI_DATA_DIR points.
volumes:
- ../TechAPI/data:/data:ro
# Seed the database, then serve. Idempotent: re-running skips existing rows.
command: >
sh -c "python -m app.seed && uvicorn app.main:app --host 0.0.0.0 --port 8000"
Expand Down
67 changes: 36 additions & 31 deletions docs/DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,43 @@
# TechAPI — development guide & handoff
# TechEngine development guide

> Read this first. Full spec: [SPEC.md](SPEC.md). Static-dump design:
> [DATA_PIPELINE.md](DATA_PIPELINE.md).

## What this is

TechAPI — a free, open REST API for consumer-electronics specs (smartphones,
SoCs, GPUs, CPUs, brands). FastAPI + SQLModel. **This repo is
the public product: the API + a curated dataset + a static JSON dump.** It does
**not** contain data-collection/scraping tooling (that is maintained separately;
see "Data scope" below).
TechEngine is the engine for the TechAPI dataset of consumer-electronics specs:
the validator, the FastAPI read API, the static JSON dump generator, the
coverage report, the collectors (`app/ingest`) and the verification tiers
(`app/verify`). **It holds no data.** The dataset lives in
[GetTechAPI/TechAPI](https://github.com/GetTechAPI/TechAPI) and is read from a
TechAPI checkout (see the note under "Repo layout").

## Current status

- **Stack**: FastAPI, SQLModel, Pydantic v2, Python 3.12. DB defaults to SQLite
(`DATABASE_URL`); Postgres 16 via `docker-compose.yml`.
- **Endpoints** (under `/v1`): `health`, `version`, `brands`(+`/{slug}` +`/{slug}/smartphones`),
`socs`(+`/{slug}` +`/{slug}/smartphones`), `smartphones`(+`/{slug}` +`/{slug}/score`),
`gpus`(+`/{slug}`), `cpus`(+`/{slug}`, `?segment=`). Docs at `/scalar` and `/docs`.
- **Endpoints** (under `/v1`): `health`, `version`, and one list + `/{slug}`
detail per collection: `brands` (+`/{slug}/smartphones`), `socs`
(+`/{slug}/smartphones`), `smartphones`, `tablets`, `watches`, `pdas`, `gpus`,
`cpus` (`?segment=`), `laptops`, `monitors`, `software`, `websites`,
`device-catalog`. `socs`, `smartphones`, `gpus` and `cpus` also have
`/{slug}/score`. Docs at `/scalar` and `/docs`.
- **Formats**: list `{count,next,previous,results}` (§7.4); error
`{error:{code,message,request_id,documentation_url}}` (§7.5); smartphone detail
per SPEC appendix C (embeds brand+soc+manufacturer + computed `score`).
- **Tests**: `pytest` is the canonical test runner. `ruff check app tests` +
`mypy app` are the static gates in CI.
- **NOT done**: cloud deploy (Railway/Fly need accounts) and public static hosting.
Dockerfile + workflows written but not executed in the cloud.
- **Hosting**: there is no hosted API server. `deploy-pages.yml` publishes the
landing site with a static dump to GitHub Pages on every push to `main`;
TechAPI builds its own copy of the dump with the engine commit it pins.

### Curated dataset (validated, what the API serves)
brand 129 · soc 123 · smartphone 367 · gpu 330 · cpu 976. **This is a curated subset,
NOT an exhaustive catalog of every device.** It is intentionally hand-verified;
breadth is expanded out-of-band (see "Data scope").
### Data scope

### Data scope (important)
This repo holds only **curated, validated** data and serves/dumps it. Bulk
collection/scraping and the pre-review candidate pool are **not part of this
repo** and are maintained through a separate pipeline. Local contributors working
in this Codespace: see `NOTES.local.md` (gitignored, not committed).
The dataset is large and changes often, so no record counts are kept here; see
TechAPI's README. Collectors live in `app/ingest` and `app/verify`. Nothing runs
on a schedule: they run when a maintainer dispatches TechAPI's `collect.yml`,
and each run uploads a patch that a maintainer turns into a TechAPI PR. Do not
run them against a TechAPI checkout you are not prepared to diff and review.

## Repo layout

Expand Down Expand Up @@ -78,11 +80,12 @@ docs/ SPEC.md, DATA_PIPELINE.md, DEVELOPMENT.md
## How to run

```bash
pip install -e ".[dev]" # deps already installed in this Codespace
python -m app.validate # check curated data
git clone https://github.com/GetTechAPI/TechAPI.git ../TechAPI # the data
pip install -e ".[dev]"
python -m app.validate # check the dataset
python -m app.seed # dataset → ./techapi.db (SQLite)
uvicorn app.main:app --reload # serve; curl http://localhost:8000/v1/smartphones/galaxy-s25
pytest -q --cov=app # tests (target >60%, currently ~98%)
pytest -q --cov=app # tests (CI fails under 60% coverage)
ruff check app tests && mypy app
python -m app.dump # generate ./dump/ static tree (gitignored)
```
Expand All @@ -95,24 +98,26 @@ python -m app.dump # generate ./dump/ static tree (gitignored)
- **Data restructured** to singular names + brand subfolders (maintainer request).
- **Static-dump pivot** — `app/dump.py` exports the API to a static JSON tree,
built by TechAPI at deploy time (see DATA_PIPELINE.md §3).
- **Scoring** is a Phase-0 reference-based approximation; Phase 1 → dataset-wide
min-max (§8.4). Raw third-party benchmarks (Geekbench/AnTuTu/Cinebench/Time Spy)
are stored as algorithm inputs but NOT exposed (ADR-006).
- **Scoring** reads its weights and reference scales from `config/scoring.yaml`
(§8). Raw third-party benchmarks (Geekbench/AnTuTu/Cinebench/Time Spy) are
stored as algorithm inputs but NOT exposed (ADR-006).

## Conventions (IMPORTANT)

- **Commits**: Conventional Commits (`feat(api): …`, `data: …`, `docs: …`).
- **Commits**: Conventional Commits (`feat(api): …`, `fix(ingest): …`, `docs: …`).
Keep commit messages clean and to the point.
- **Data accuracy**: only real, sourced models. All brand, device, and chip
records require `source_urls`. Do not fabricate unsourced "all devices" data
(SPEC §1.6).
- 100% type hints; tests alongside code; new top-level data fields must match the
- 100% type hints; tests under `tests/`; new top-level data fields must match the
SQLModel (seeder does `Model(**record)` — unknown keys break it).

## Gotchas

- SQLModel `table=True` models skip Pydantic validation, so `seed.py` coerces
`release_date` strings to `date` objects manually.
- `app/services/scoring.py` camera score is intentionally simple (a 50MP flagship
can score below a 200MP phone) — known Phase-0 limitation, not a data bug.
- Nothing is committed yet — `git status` shows the build as adds.
- The camera score in `app/services/scoring/phones.py` is intentionally simple
(a 50MP flagship can score below a 200MP phone): a known limitation, not a
data bug.
- `app.seed` reads record timestamps from the TechAPI checkout's git history; a
shallow clone or a plain copy of `data/` falls back to the seed time.
4 changes: 2 additions & 2 deletions site/astro.config.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ import { defineConfig } from 'astro/config';
// GitHub Pages project site. Data dump (v1/, openapi.json) is copied alongside
// the built site in CI, so internal links use import.meta.env.BASE_URL.
export default defineConfig({
site: 'https://seungpyo1007.github.io',
base: '/TechAPI',
site: 'https://gettechapi.github.io',
base: '/TechEngine',
});
Loading
Loading