view a parquet file in PyCharm or IntelliJ IDEA

A columnar file keeps its values by column and its map in a footer at the end, so the schema can be read before a single row is touched. PyCharm and IntelliJ IDEA have four ways in, and the last two below, code and a viewer plugin such as Parquet Lens, work on any edition.

the data loader, built into IntelliJ IDEA

Since 2024.1, IntelliJ IDEA with Database Tools shows a .parquet file in its data editor through a read-only data loader, with no plugin to install. JetBrains' help page for the loaders is marked Ultimate and says Database Tools support is limited in IntelliJ IDEA without the Ultimate subscription.

JetBrains' plugin, on IntelliJ IDEA and PyCharm Pro

JetBrains makes Big Data File Viewer, which shows a .parquet file as a table. Its Marketplace listing names eleven IDEs, IntelliJ IDEA and PyCharm Pro among them, and no Community edition. On one of those, install it under Settings, Plugins, Marketplace, then double-click the file in the Project tool window.

code, on any edition

The answer usually given for the free editions is two lines of code: import pandas as pd, then df = pd.read_parquet("data.parquet"). With PyArrow under it, pandas reads the file into a frame, and the debugger shows that frame. It costs nothing. By default, though, the whole file goes into memory, every row of every column.

a viewer plugin, on any edition

Avro and Parquet Viewer, a free plugin from another developer, adds a tool window that takes a dropped file and shows its records as a table or as JSON beside the schema.

Parquet Lens opens a .parquet file in an editor tab, straight off disk, with no notebook and no conversion step. It runs in IntelliJ IDEA, PyCharm and every other JetBrains IDE on 2024.2 or newer, Community editions included, Android Studio from Ladybug.

The same plugin opens .avro containers, TFRecord datasets plain or compressed, and Arrow IPC and Feather v2 files.

installing and opening

Install Parquet Lens from the JetBrains Marketplace and open the file from the project tree. The Schema tab is up before a row decodes, carrying the logical and physical type of every column, its codec and encodings, and the null count with the minimum and maximum the footer stores per column. Rows then arrive 100 to a page, and the Page size box goes up to 5000. Go-to-row lands on any row you name, and Ctrl+F streams a text search across every row.

Parquet Lens in IntelliJ IDEA: a paged data table with Schema and Page stats tabs and a row detail pane — sensors.parquet in a JetBrains IDE: 2,400 pages of 100 rows, the Schema and Page stats tabs beside the data, the selected row in the detail pane.

the values you will see

Timestamps keep their unit and timezone, decimals and int64 keep their precision, and a nested list or struct or map renders as JSON with a pretty detail pane beside it. A corrupt or truncated file reports its reason, and a codec the reader does not have is refused by name.

free, and what Pro adds

Reading is free. A row copies out as JSON and the page as CSV. Pro adds a filter such as lang == "en" && score >= 0.5 evaluated across every row group with min/max pruning where the writer stored those statistics, a sort of the whole dataset by any column, exact distinct counts over every row, and an export of all rows or the matches to CSV, JSONL or a new Parquet file.

The viewer is free and stays free. Pro is a subscription for this plugin: USD 2.90 a month or USD 29 a year for individual use. For organizations it is, per user, USD 5.80 a month or USD 58 a year.

The file is never written to; a Pro export creates a new file. Parquet Lens is not a query engine either: the Pro filter and the whole-file sort, no SQL and no joins across files. Suggestions go to support@twilightventures.dev, and Report a Parquet Lens Problem, at the end of the Help menu, opens a prefilled new issue in your browser for you to read before you post.

outside a JetBrains IDE

The same files open in Parquet Lens for VS Code; a Parquet file also opens as an attachment on a Confluence page or a Jira issue with Data Lens.

Install from the JetBrains Marketplacefree to read; the Pro trial runs 30 days