Fraud detection workflow example

[jayavenkatesh19]

Towards #667

Adds a new workflow example demonstrating how to wrap the NVIDIA Financial Fraud Detection AI
Blueprint in production infrastructure
using Prefect (orchestration), MLflow (experiment tracking + model registry), and Triton (champion/challenger
serving with native versioning).

The notebook walks through the architecture, each pipeline stage, running and monitoring the pipeline, and
scaling to multiple machines.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[ncclementi]

I've only taken a quick look, but I noticed that the instructions are quite generic, and I think it would be helpful if we choose a provider (AWS, Brev or whatever is easier) and have the end to end example shown.

Currently, when I read it I'm no sure the user will be quite sure where to start or how to set things up.

How did you set up things and do the running?

[ncclementi]

docker compose up -d starts only MLflow and Prefect. Use it for preprocessing and training work that does not need Triton:

$ docker compose up -d

docker compose --profile gpu up -d starts MLflow, Prefect, and Triton. Use it before evaluation, deployment, or the full end-to-end pipeline:

$ docker compose --profile gpu up -d

This is a bit repetitive, I'd write something like:

If you want to do preprocessing and training work that does not need Triton, you can start a container only with MLFlow and Prefect:

docker compose up -d

But if you want to run the end-to-end pipeline, you'll need MLflow, Prefect, and Triton (or something like this )

docker compose --profile gpu up -d

Other questions:

• why would someone want to only do the first case?
• I didn't understand what did you mean with "Use it before evaluation, deployment"

Reply via ReviewNB

[jayavenkatesh19]

Changed the wording on this to reflect using

docker compose --profile gpu up -d

as the default.

[ncclementi]

I wonder if this is a bit distracting here, I appreciate the explanations but I think it might be more important to share the monitoring and diagnostic section as well as the scaling.

What do you think about putting a note that says something like "If you are familiarized with this you can to skip directly to the monitoring section and put a link to it?

Reply via ReviewNB

[jayavenkatesh19]

Moved the monitoring section above this, and explanations occur before the scaling section.

[ncclementi]

Local machine: runs Prefect and MLflow servers, providing direct browser access to both UIs without port forwarding. Used only for monitoring and triggering runs.

Does this machine needs to have a GPU? We should clarify this.

Then, do you need to start from scratch or how do you go from what we just run to this?

DO you have to have the triton container running in brev?

Point the GPU instance at the local machine by setting the API URLs:

When you are doing this, you have this note

"The GPU instance must have network access to the local machine on ports 4200 (Prefect) and 5050 (MLflow). Ensure these ports are open in your firewall or security group rules."

We should share how to do this on brev, is it possible to show that?

Reply via ReviewNB

[jayavenkatesh19]

This isn't a scenario we can show on Brev, but rather on how this example translates to production grade MLOps infrastructure setups.

In these scenarios, data scientists/labs have a centralized MLFlow/Prefect servers that they use to track multiple experiments and results. In this section, we are just showing that our infrastructure design is compatible with such a system, and as long as you have a Prefect GPU worker, you can trigger experiments and log metrics from this centralized orchestration mechanism without the need for having a GPU locally.

Brev cannot access ports on your laptop without reverse tunnelling so we can't show this setup. However, I did make it clear that the earlier setup is the recommended path on Brev and what are the requirements to use a split machine mode.

[ncclementi]

Great work @jayavenkatesh19 this looks great!