Live: Python in WebAssembly
Run the repo's pandas profiler on NYC Open Data
This code is the Spark profiler's test oracle. It infers each column's type from parse ratios and adds semantic types and data-quality flags. On the bundled datasets its output is checked field by field against the profile Spark committed to the repo.
Downloads Python 3.13 (Pyodide 0.29.5) with numpy and pandas: about 13 MB from cdn.jsdelivr.net, cached after the first run.
Recorded output, not generated here. qwen3:14b, run locally with Ollama, wrote a user-focused (UFD) and a search-focused (SFD) description for each of the 100 datasets from its profile alone, never seeing the portal's text. These are the files in the repo's descriptions/, with the scores from results/.
Dataset search with BM25
Recall@1 over the 100 datasets when each variant is indexed. Portal-tag queries are built from the portal's own tags with no LLM; the other set was written by an LLM from the original descriptions, so it favours them. Source: results/eval_search.csv.
Recorded timings, not run here. Medians over the repeats in the repo's results/bench.csv, measured on a 14-core M4 Pro (24 GB) by a harness that waits for a quiet machine and logs the load on every row. Your browser's timings in the Profiler tab are not comparable.