Skip to content

Conversation

@JoshD94
Copy link
Contributor

@JoshD94 JoshD94 commented Jan 30, 2026

Overview

Updates to the scraping logic, new integrations for daily sun content, and other fixes.

Changes Made

  • Daily Sun Integration: Implemented scraping and display for Daily Sun articles and images.

  • Basketball Support: Added scraping for basketball score breakdowns and box scores.

  • Media Enhancements: Added ticket links to game objects and duration/sport type fields to YouTube video models.

  • Duplicate Prevention: Implemented logic to detect and prevent duplicate games during scraping.

  • Game Status: Added handling for tournaments and TBD/TBA game updates.

  • Filtering & Metadata: Added "find by date" query support for advanced frontend filters and standardized article dates to ISO 8601.

  • Server Config: Fixed Gunicorn argument parsing, initialization order, and updated the service port to 8000 (previously 8001).

  • Database: Resolved MongoDB _id field errors and removed verbose logging.

Test Coverage

Next Steps (delete if not applicable)

Related PRs or Issues (delete if not applicable)

Screenshots (delete if not applicable)

Screen Shot Name

mateow99 and others added 30 commits March 24, 2025 20:23
Fix duplicate games and fetching logic
This reverts commit 721f0f8, reversing
changes made to bd14659.
implement articles from daily sun
- Fix argument parsing to only run when script is executed directly, not when imported by gunicorn
- Fix MongoDB _id field modification error in Daily Sun scraper by removing _id from upsert operations
- Maintain all functionality while eliminating error messages
Move signal handlers after argument parsing to fix initialization order
Added ticket links to game objects
TushigBili and others added 18 commits October 22, 2025 18:17
- Updated YoutubeVideo model to include duration attribute.
- Modified CreateYoutubeVideo mutation to accept duration as an argument.
- Implemented get_video_duration function to fetch video duration from YouTube API.
- Updated process_video_item to include video duration in the processed data.
- Adjusted YoutubeVideoType to reflect the new duration field.
Added find by date query to support advanced filters for frontend
Add duration field to YoutubeVideo model and related mutations
Refactor article date handling to use ISO 8601 format
Implement sport type extraction from article titles
@AnikDey-exe
Copy link

this looks good to me

Copy link
Contributor

@claiireyu claiireyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

JoshD94 and others added 2 commits January 30, 2026 14:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants