{"name":"huggingface-datasets","url":"https://skills.sh/huggingface/skills/huggingface-datasets","install":"npx skills add huggingface/skills --skill huggingface-datasets","sdk":"huggingface","key":"huggingface/huggingface-datasets","description":"Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.","hasContent":true,"content":"---\r\nname: huggingface-datasets\r\ndescription: Use this skill for Hugging Face Dataset Viewer API workflows that fetch subset/split metadata, paginate rows, search text, apply filters, download parquet URLs, and read size or statistics.\r\n---\r\n\r\n# Hugging Face Dataset Viewer\r\n\r\nUse this skill to execute read-only Dataset Viewer API calls for dataset exploration and extraction.\r\n\r\n## Core workflow\r\n\r\n1. Optionally validate dataset availability with `/is-valid`.\r\n2. Resolve `config` + `split` with `/splits`.\r\n3. Preview with `/first-rows`.\r\n4. Paginate content with `/rows` using `offset` and `length` (max 100).\r\n5. Use `/search` for text matching and `/filter` for row predicates.\r\n6. Retrieve parquet links via `/parquet` and totals/metadata via `/size` and `/statistics`.\r\n\r\n## Defaults\r\n\r\n- Base URL: `https://datasets-server.huggingface.co`\r\n- Default API method: `GET`\r\n- Query params should be URL-encoded.\r\n- `offset` is 0-based.\r\n- `length` max is usually `100` for row-like endpoints.\r\n- Gated/private datasets require `Authorization: Bearer <HF_TOKEN>`.\r\n\r\n## Dataset Viewer\r\n\r\n- `Validate dataset`: `/is-valid?dataset=<namespace/repo>`\r\n- `List subsets and splits`: `/splits?dataset=<namespace/repo>`\r\n- `Preview first rows`: `/first-rows?dataset=<namespace/repo>&config=<config>&split=<split>`\r\n- `Paginate rows`: `/rows?dataset=<namespace/repo>&config=<config>&split=<split>&offset=<int>&length=<int>`\r\n- `Search text`: `/search?dataset=<namespace/repo>&config=<config>&split=<split>&query=<text>&offset=<int>&length=<int>`\r\n- `Filter with predicates`: `/filter?dataset=<namespace/repo>&config=<config>&split=<split>&where=<predicate>&orderby=<sort>&offset=<int>&length=<int>`\r\n- `List parquet shards`: `/parquet?dataset=<namespace/repo>`\r\n- `Get size totals`: `/size?dataset=<namespace/repo>`\r\n- `Get column statistics`: `/statistics?dataset=<namespace/repo>&config=<config>&split=<split>`\r\n- `Get Croissant metadata (if available)`: `/croissant?dataset=<namespace/repo>`\r\n\r\nPagination pattern:\r\n\r\n```bash\r\ncurl \"https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=0&length=100\"\r\ncurl \"https://datasets-server.huggingface.co/rows?dataset=stanfordnlp/imdb&config=plain_text&split=train&offset=100&length=100\"\r\n```\r\n\r\nWhen pagination is partial, use response fields such as `num_rows_total`, `num_rows_per_page`, and `partial` to drive continuation logic.\r\n\r\nSearch/filter notes:\r\n\r\n- `/search` matches string columns (full-text style behavior is internal to the API).\r\n- `/filter` requires predicate syntax in `where` and optional sort in `orderby`.\r\n- Keep filtering and searches read-only and side-effect free.\r\n\r\nFor CLI-based parquet URL discovery or SQL, use the `hf-cli` skill with `hf datasets parquet` and `hf datasets sql`.\r\n\r\n## Creating and Uploading Datasets\r\n\r\nUse one of these flows depending on dependency constraints.\r\n\r\nZero local dependencies (Hub UI):\r\n\r\n- Create dataset repo in browser: `https://huggingface.co/new-dataset`\r\n- Upload parquet files in the repo \"Files and versions\" page.\r\n- Verify shards appear in Dataset Viewer:\r\n\r\n```bash\r\ncurl -s \"https://datasets-server.huggingface.co/parquet?dataset=<namespace>/<repo>\"\r\n```\r\n\r\nLow dependency CLI flow (`npx @huggingface/hub` / `hfjs`):\r\n\r\n- Set auth token:\r\n\r\n```bash\r\nexport HF_TOKEN=<your_hf_token>\r\n```\r\n\r\n- Upload parquet folder to a dataset repo (auto-creates repo if missing):\r\n\r\n```bash\r\nnpx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data\r\n```\r\n\r\n- Upload as private repo on creation:\r\n\r\n```bash\r\nnpx -y @huggingface/hub upload datasets/<namespace>/<repo> ./local/parquet-folder data --private\r\n```\r\n\r\nAfter upload, call `/parquet` to discover `<config>/<split>/<shard>` values for querying with `@~parquet`.\r\n\r\n## Agent Traces\r\n\r\nThe Hub supports raw agent session traces from Claude Code, Codex, and Pi Agent. Upload them to Hugging Face Datasets as original JSONL files and the Hub can auto-detect the trace format, tag the dataset as `Traces`, and enable the trace viewer for browsing sessions, turns, tool calls, and model responses. Common local session directories:\r\n\r\n- Claude Code: `~/.claude/projects`\r\n- Codex: `~/.codex/sessions`\r\n- Pi: `~/.pi/agent/sessions`\r\n\r\nDefault to private dataset repos because traces can contain prompts, file paths, tool outputs, secrets, or PII. Preserve the raw `.jsonl` files and nest them by project/cwd instead of uploading every session at the dataset root.\r\n\r\n```bash\r\nhf repos create <namespace>/<repo> --type dataset --private --exist-ok\r\nhf upload <namespace>/<repo> ~/.codex/sessions codex/<project-or-cwd> --type dataset\r\n```\r\n","contentSource":"skills.sh/api/download/huggingface/skills/huggingface-datasets","contentFetchedAt":"2026-07-27T08:59:35.684Z"}