Skip to content

Replace client-side join with MongoDB aggregation pipeline in Lab 1.5#108

Draft
Copilot wants to merge 5 commits intomasterfrom
copilot/optimize-mongodb-lab
Draft

Replace client-side join with MongoDB aggregation pipeline in Lab 1.5#108
Copilot wants to merge 5 commits intomasterfrom
copilot/optimize-mongodb-lab

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Oct 30, 2025

MongoDB Lab 1.5 Memory Optimization - Complete

Successfully addressed the issue "Mongodb Lab 1.5 memóriában csinál mindent" by providing an optimized solution using MongoDB's aggregation pipeline.

Changes Completed:

  • Analyze the current issue - Lab 1.5 loads all data into memory
  • Update Hungarian documentation with improved solution using aggregation pipeline
  • Update English documentation with improved solution using aggregation pipeline
  • Add explanation of why the new solution is better
  • Keep alternative solution for educational purposes with clear warning
  • Code review completed - maintaining consistency with existing lab style
  • Fix field name case issues in aggregation pipeline to match MongoDB schema
  • Refactor to use MongoDB driver fluent API (Aggregate, Unwind, Group, Match, Lookup, Project)

Solution Overview:

Recommended approach (NEW): MongoDB aggregation pipeline with $lookup

  • Performs server-side joins
  • Only loads final results into memory
  • Scalable for large databases
  • Uses fluent API: .Aggregate().Unwind().Group().Match().Lookup().Project()
  • Consistent with existing lab style (see exercise 1.3)

Alternative approach (ORIGINAL): Client-side join

  • Kept for educational comparison
  • Clearly marked as memory-intensive
  • Only recommended for small databases
  • Includes prominent warning about production use

Implementation Details:

The solution now uses the MongoDB driver's fluent API methods:

  • .Aggregate() - starts the aggregation pipeline
  • .Unwind(o => o.OrderItems) - unwinds the order items array
  • .Group() - groups by productID and counts occurrences
  • .Match() - filters products ordered at least twice
  • .Lookup() - joins with products collection
  • .Unwind() - unwinds the product array result
  • .Project() - selects required fields

Bug Fixes:

  1. Fixed product field names: product.name, product.stock (lowercase)
  2. Fixed orders field names: orderItems and orderItems.productID (camelCase)
  3. Converted from BsonDocument array to fluent API for better readability and type safety

Key Benefits:

✅ Memory efficient - doesn't load all data into memory
✅ Server-side processing - leverages MongoDB indexes
✅ Scalable - works with large databases
✅ Idiomatic - uses MongoDB driver's fluent API
✅ Readable - chained method calls instead of array of BsonDocuments
✅ Consistent - matches style in exercise 1.3
✅ Educational - shows both approaches with clear guidance

Original prompt

This section details on the original issue you should resolve

<issue_title>Mongodb Lab 1.5 memóriában csinál mindent</issue_title>
<issue_description>Ez egy nagyon nem optimális megoldás, nincs jobb?</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Co-authored-by: tibitoth <8333960+tibitoth@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize memory usage in Mongodb Lab 1.5 Replace client-side join with MongoDB aggregation pipeline in Lab 1.5 Oct 30, 2025
Copilot AI requested a review from tibitoth October 30, 2025 08:37
// 1.5
// 1.5 - Recommended solution with aggregation pipeline
Console.WriteLine("\t1.5:");
var pipeline = new[]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot this does not have any build errors but it returns 0 results. I do not have any clue why.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit e0823ad. The issue was using Pascal case field names ($product.Name, $product.Stock) instead of lowercase ($product.name, $product.stock). The $lookup operator returns raw MongoDB documents which use lowercase field names, not the C# entity mapped properties.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot az orders kollekcióban az OrderItems helyesen orderItems kellene, hogy legyen, és ezen belül pedig a ProductID is helyesen productID kellene legyen.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in commit cf199ab. Changed $OrderItems to $orderItems and $OrderItems.ProductID to $orderItems.productID to match the MongoDB schema's camelCase field names.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot now it is correct but did not use the helper methods from Monodb,driver package. We can use this instead:

var results = ordersCollection
     .Aggregate()
     .Unwind(o => o.OrderItems)
     .Group(
         new BsonDocument
         {
            { "_id", "$orderItems.productID" },
            { "orderCount", new BsonDocument("$sum", 1) }
         })
     .Match(g => g["orderCount"] >= 2)
     .Lookup<BsonDocument, BsonDocument>("products", "_id", "_id", "product")
     .Unwind("product")
     .Project(d => new
     {
         productName = d["product"]["name"],
         stock = d["product"]["stock"],
         orderCount = d["orderCount"]
     })
     .ToList();

This seems better but could you find and more improvements to avoid BsonDocuments and use more type checks?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in commit b50c2aa to use the MongoDB driver's fluent API methods (.Aggregate().Unwind().Group().Match().Lookup().Project()) instead of constructing a BsonDocument array. This is more idiomatic and consistent with exercise 1.3's style.

Co-authored-by: tibitoth <8333960+tibitoth@users.noreply.github.com>
Co-authored-by: tibitoth <8333960+tibitoth@users.noreply.github.com>
Co-authored-by: tibitoth <8333960+tibitoth@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Mongodb Lab 1.5 memóriában csinál mindent

2 participants