Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 49 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,68 @@
# data-analyses

Place for sharing quick reports, and works in progress

This repository is for quick sharing of works in progress and simple analyses.
For collaborative short-term tasks, create a new folder and work off a separate branch.
For longer-term projects, consider making a new repository!


## Using this Repo

* Use [this link](https://docs.calitp.org/data-infra/analytics_tools/saving_code.html#onboarding-setup) to get started in JupyterHub, set up SSH, and start commiting to the repo!


### JupyterHub Developers

If you are developing in JupyterHub, follow the [JupyterHub setup docs](https://docs.calitp.org/data-infra/analytics_tools/jupyterhub.html).


### Contributing
#### Pre-commit

This repository uses pre-commit hooks to format code, including [Black](https://black.readthedocs.io/en/stable/index.html). This ensures baseline consistency in code formatting.
Follow these steps to start contributing:

1. [Clone](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) this `data-analyses` repo.

2. From the repo root (`data-analyses/`), run `make install_env` (runs `uv sync` + pre-commit setup)

3. In JupyterHub, select the **"Pyproject Local"** kernel when opening a notebook

> [!NOTE]
> If you run into the error `No such file or directory`, you may need to [install uv](https://docs.astral.sh/uv/getting-started/installation/) running `pip install uv`.

### uv

This repository uses uv for package management. To learn more go to [uv documentation](https://docs.astral.sh/uv/concepts/projects/dependencies/).

> [!IMPORTANT]
> Before contributing to this project, please install dependencies and pre-commit by running `make install_env` in the root of the repo.
Basic commands:

Once installed, pre-commit checks will run before you can make commits locally. If a pre-commit check fails, it will need to be addressed before you can make your commit. Many formatting issues are fixed automatically within the pre-commit actions, so check the changes made by pre-commit on failure -- they may have automatically addressed the issues that caused the failure, in which case you can simply re-add the files, re-attempt the commit, and the checks will then succeed.
* `uv sync` install missing packages, update existing ones, and remove unnecessary ones to ensure the environment matches the lockfile.
* `uv add <package name>` include and install a new package to the main project.
* `uv add <package name> --dev` include and install new packages/dependencies to the `dev` group.
* `uv add <package name> --portfolio` include and install new packages/dependencies to the `portfolio` group.
* `uv add <package name> --test` include and install new packages/dependencies used only for testing under the `test` group.
* `uv remove <package name>` remove and uninstall packages/dependencies from the project.

### nbdime

[`nbdime`](https://github.com/jupyter/nbdime) provides command-line tools for diffing and merging notebooks.

Basic commands:

* `nbdiff` compare notebooks in a terminal-friendly way.
* `nbshow` present a single notebook in a terminal-friendly way.

### Pre-commit

This repository uses pre-commit hooks to format code, including [Black](https://black.readthedocs.io/en/stable/index.html). This ensures baseline consistency in code formatting.

Pre-commit checks will run before you can make commits locally. If a pre-commit check fails, it will need to be addressed before you can make your commit.
Many formatting issues are fixed automatically within the pre-commit actions, so check the changes made by pre-commit on failure -- they may have automatically addressed the issues that caused the failure, in which case you can simply re-add the files, re-attempt the commit, and the checks will then succeed.

Installing pre-commit locally saves time dealing with formatting issues on pull requests. There is a [GitHub Action](./.github/workflows/lint.yml)
that runs pre-commit on all files, not just changed ones, as part of our continuous integration.


## Quick Links - Get Started in Data Analysis

#### Data Analytics Documentation - Welcome
Expand All @@ -32,14 +73,17 @@ https://docs.calitp.org/data-infra/analytics_welcome/overview.html

https://docs.calitp.org/data-infra/analytics_tools/overview.html


## Publishing Reports

[The sites folder](./portfolio/sites/) contains the YAML files that drive sites
deployed to [https://analysis.calitp.org/](https://analysis.calitp.org/); the existing
sites can be used as examples/templates for deploying additional sites. Also, the
Data Services Documentation has a [specific chapter](https://docs.calitp.org/data-infra/publishing/overview.html)
dedicated to various ways to publish data.

### Caveats (when using the portfolio site)

Jupyter Book/Sphinx do not play nicely with Markdown headers written out in `display()`
calls. Therefore, [portfolio.py](./portfolio.py) uses a custom Papermill
engine to template Markdown cells directly, following Python formatted-string
Expand Down
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@
name = "cal-itp-data-analyses"
version = "0.1.0"
requires-python = ">=3.11.0, <3.12.0"
dependencies = [
"nbdime>=4.0.4",
]

[tool.uv.workspace]
members = [
Expand Down
56 changes: 56 additions & 0 deletions uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading