Pasindu: Route Optimizer Module#41
Open
L0rd008 wants to merge 16 commits into
Open
Conversation
… Variables This pull request addresses several issues with the route optimizer service: 1. Google Maps API Key Configuration: - Modified the settings module to handle missing API keys gracefully - Added fallback to dummy keys for testing environments - Implemented proper environment variable loading 2. Environment Variable Setup: - Added support for loading configuration from env_var.env file - Created clear documentation for required environment variables 3. Important Note: - The Google Maps API key should be added to the env_var.env file in the root directory - Format: `GOOGLE_MAPS_API_KEY=your_api_key_here` - The main error in the test output was related to the missing Google Maps API key, which was causing a ValueError in route_optimizer/settings.py 4. What This PR Does Not Change: - No modifications to the pathfinding logic or algorithms - Core optimization functionality remains the same
This is not yet complete. Want to use this save as a checkpoint.
There is still to complete. This is a checkpoint.
Still in the fixing state, currently having a power cut, hence adding PR to save work.
Okay, I understand! You want to use the detailed breakdown of the `route_optimizer` module that we just discussed as the description for your GitHub Pull Request. That's a great way to provide comprehensive context for your reviewers.
Here's the description, formatted and ready for your PR:
---
## Pull Request: Route Optimizer Module - Full Functionality Overview
This Pull Request introduces the complete `route_optimizer` module, a Django application designed for sophisticated route planning and optimization.
### Overview
The **Route Optimizer** module is a comprehensive Django app designed to calculate optimal routes for a fleet of vehicles to make deliveries (or pickups) to a set of locations. It considers various constraints such as vehicle capacities, time windows, traffic conditions, and operational costs. The module can use either local calculations (like Haversine for distance) or external APIs (like Google Maps Distance Matrix API) for more accurate real-world data. It also supports dynamic rerouting in response to real-time events.
---
### Core Files (`route_optimizer/core/`)
The `core` directory contains the fundamental algorithms, data type definitions, and constants that form the backbone of the route optimization logic.
#### 1. `constants.py` ([Logistics\route_optimizer\core\constants.py](file:///Logistics\route_optimizer\core\constants.py))
* **Functionality**:
* Defines various constants used throughout the optimization process.
* Includes scaling factors (e.g., `DISTANCE_SCALING_FACTOR`, `CAPACITY_SCALING_FACTOR`, `TIME_SCALING_FACTOR`) required by OR-Tools to work with integer arithmetic.
* Specifies safety bounds for distance and time values (e.g., `MAX_SAFE_DISTANCE`, `MAX_SAFE_TIME`).
* **Important Points**:
* **Consistency is Key**: Ensure these constants are consistently used and understood across all modules, especially between the main code and tests. Mismatches in values like `MAX_SAFE_DISTANCE` have caused issues previously.
* **Scaling Factor Impact**: The scaling factors directly affect the precision and behavior of the OR-Tools solver. Adjustments might be needed based on the typical range of input values.
* The commented-out section suggests different scaling strategies might have been considered; the current active set is what's in use.
#### 2. `dijkstra.py` ([Logistics\route_optimizer\core\dijkstra.py](file:///Logistics\route_optimizer\core\dijkstra.py))
* **Functionality**:
* Provides an implementation of Dijkstra's algorithm.
* `DijkstraPathFinder` class offers:
* `calculate_shortest_path(graph, start, end)`: Finds the shortest path between a single pair of nodes.
* `calculate_all_shortest_paths(graph, nodes)`: Calculates shortest paths between all specified pairs of nodes.
* `_validate_non_negative_weights(graph)`: Ensures no negative edge weights.
* **Important Points**:
* **Negative Weights**: Raises a `ValueError` for negative weights. If negative weights are needed, an alternative like Bellman-Ford would be required.
* **Graph Representation**: Expects graph as a dictionary of dictionaries (adjacency list with weights).
* **Use Case**: Used by `PathAnnotator` for detailed path segments when not using external APIs.
#### 3. `distance_matrix.py` ([Logistics\route_optimizer\core\distance_matrix.py](file:///Logistics\route_optimizer\core\distance_matrix.py))
* **Functionality**:
* `DistanceMatrixBuilder` class for creating and managing distance matrices.
* Supports Haversine, Euclidean, and Google Maps Distance Matrix API calculations.
* Includes caching (`DistanceMatrixCache` model), API request retries, address formatting, response processing, matrix sanitization (`_sanitize_distance_matrix`), traffic factor application (`add_traffic_factors`, `_apply_traffic_safely`), and matrix-to-graph conversion.
* **Important Points**:
* **API Key**: Relies on `GOOGLE_MAPS_API_KEY` from `settings.py`.
* **API Quotas**: Caching is vital to manage API usage.
* **Fallback Behavior**: Falls back to Haversine if API fails.
* **Sanitization**: `_sanitize_distance_matrix` is crucial for clean numerical data, replacing `NaN`, `inf`, etc., with `MAX_SAFE_DISTANCE` or 0.
* **Traffic Application**: `_apply_traffic_safely` includes bounds checking for traffic factors.
#### 4. `ortools_optimizer.py` ([Logistics\route_optimizer\core\ortools_optimizer.py](file:///Logistics\route_optimizer\core\ortools_optimizer.py))
* **Functionality**:
* `ORToolsVRPSolver` class for solving Vehicle Routing Problems (VRP) using Google OR-Tools.
* `solve(...)`: Basic VRP with capacity constraints.
* `solve_with_time_windows(...)`: VRP with time window constraints.
* Handles vehicle start/end locations, capacities, and cost minimization.
* **Important Points**:
* **Integer Scaling**: Critical for OR-Tools; uses scaling factors from `constants.py`.
* **Depot Handling**: `depot_index` is fundamental.
* **Callbacks**: Distance, demand, and time callbacks are essential.
* **Solution Interpretation**: Output is parsed into `OptimizationResult`.
* **Time Limits**: Configurable `time_limit_seconds`.
* **Empty Problem**: Creates depot-to-depot routes if no deliveries.
#### 5. `types_1.py` ([Logistics\route_optimizer\core\types_1.py](file:///Logistics\route_optimizer\core\types_1.py))
* **Functionality**:
* Defines core Data Transfer Objects (DTOs) using `dataclass`:
* `Location`: Geographic point with coordinates, depot status, time windows, service time.
* `OptimizationResult`: Standardized output format.
* `RouteSegment`: Details of a path segment.
* `DetailedRoute`: Comprehensive vehicle route description.
* `ReroutingInfo`: Information for rerouting operations.
* `validate_optimization_result(result)`: Validates `OptimizationResult` structure.
* **Important Points**:
* **Standardization**: Ensures consistent data handling.
* **Validation**: `validate_optimization_result` is key for data integrity.
* **Mutability**: Dataclasses are mutable by default.
---
### Services Files (`route_optimizer/services/`)
The `services` directory orchestrates core logic and handles higher-level tasks.
#### 1. `depot_service.py` ([Logistics\route_optimizer\services\depot_service.py](file:///Logistics\route_optimizer\services\depot_service.py))
* **Functionality**:
* `DepotService` class for depot location utilities.
* `get_nearest_depot(locations)`: Identifies a depot. Defaults to the first depot found or the first location.
* `find_depot_index(locations)`: Returns index of the depot. Defaults to 0.
* **Important Points**:
* **Depot Assumption**: Simple logic for multiple depots (returns first). Fallback to the first location if no explicit depot.
#### 2. `external_data_service.py` ([Logistics\route_optimizer\services\external_data_service.py](file:///Logistics\route_optimizer\services\external_data_service.py))
* **Functionality**:
* `ExternalDataService` for fetching external data (traffic, weather, roadblocks).
* Currently provides mock data if `use_mocks` is true or real APIs are unimplemented.
* Includes helpers for mock data generation and combining factors.
* **Important Points**:
* **Mock Data**: Real API integrations needed for production.
* **API Keys**: Would require key management if real APIs were used.
#### 3. `optimization_service.py` ([Logistics\route_optimizer\services\optimization_service.py](file:///Logistics\route_optimizer\services\optimization_service.py))
* **Functionality**:
* `OptimizationService`: Main orchestrator for route optimization.
* `optimize_routes(...)`: Primary method. Validates inputs, creates/sanitizes distance matrix, applies traffic, determines depot, calls VRP solver, converts/enriches result with detailed paths and summary statistics.
* Helper methods for input validation, path/stats addition, result conversion, matrix sanitization.
* **Important Points**:
* **Central Orchestrator**: Ties many components together.
* **Error Handling**: General `try-except` and specific input validation.
* **API Usage Control**: `use_api` flag and `USE_API_BY_DEFAULT` setting.
* **Result Enrichment**: Multi-step process for detailed results.
* **Backward Compatibility**: `_add_detailed_paths` handles `dict` and `OptimizationResult`.
#### 4. `path_annotation_service.py` ([Logistics\route_optimizer\services\path_annotation_service.py](file:///Logistics\route_optimizer\services\path_annotation_service.py))
* **Functionality**:
* `PathAnnotator` class for adding detailed segment-by-segment path information.
* `annotate(result, graph_or_matrix)`: Uses a `path_finder` (e.g., `DijkstraPathFinder`) for segment details.
* Accepts graph or distance matrix. Handles `dict` and `OptimizationResult`.
* `_add_summary_statistics` helper ensures `detailed_routes` structure.
* **Important Points**:
* **Dependency**: Relies on an injected `path_finder`.
* **Error Handling**: Logs errors and adds placeholders for failed path calculations.
#### 5. `rerouting_service.py` ([Logistics\route_optimizer\services\rerouting_service.py](file:///Logistics\route_optimizer\services\rerouting_service.py))
* **Functionality**:
* `ReroutingService` for dynamic route adjustments.
* Methods: `reroute_for_traffic`, `reroute_for_delay`, `reroute_for_roadblock`.
* Helpers: `_get_remaining_deliveries`, `_update_vehicle_positions`.
* Relies on `OptimizationService` for re-optimization.
* **Important Points**:
* **State Management**: Accurate current state (completed deliveries, vehicle positions) is crucial; current helpers are placeholders.
* **Complexity**: Rerouting triggers a new, potentially intensive optimization.
#### 6. `route_stats_service.py` ([Logistics\route_optimizer\services\route_stats_service.py](file:///Logistics\route_optimizer\services\route_stats_service.py))
* **Functionality**:
* `RouteStatsService` calculates and adds statistics to the optimization result.
* `add_statistics(result, vehicles)`: Calculates vehicle/total costs, aggregates total stops/distance/vehicles used. Handles `OptimizationResult` and `dict`.
* **Important Points**:
* **Cost Calculation**: Uses `fixed_cost` and `cost_per_km` from `Vehicle` objects.
* **Data Dependency**: Needs `detailed_routes` with segment distances for accurate costs.
#### 7. `traffic_service.py` ([Logistics\route_optimizer\services\traffic_service.py](file:///Logistics\route_optimizer\services\traffic_service.py))
* **Functionality**:
* `TrafficService` for traffic-related information.
* `apply_traffic_factors(...)`: Wraps `DistanceMatrixBuilder.add_traffic_factors`.
* `create_road_graph(locations)`: Creates a road network graph. Potential integration point for Google Maps API for accurate topology if API key is used. Currently basic.
* **Important Points**:
* **API Integration**: `create_road_graph` is key for potential Google Maps API use for detailed pathing when `OptimizationService`'s `use_api_flag` is true.
#### 8. `vrp_solver.py` ([Logistics\route_optimizer\services\vrp_solver.py](file:///Logistics\route_optimizer\services\vrp_solver.py))
* **Functionality**:
* Contains a standalone `solve_with_time_windows(...)` function, similar to the method in `core/ortools_optimizer.py`.
* **Important Points**:
* **Redundancy/Placement**: May need refactoring. `core/ortools_optimizer.py` should be the primary OR-Tools solver implementation. This might be legacy or a specialized helper.
---
### Settings File (`route_optimizer/settings.py`)
* [Logistics\route_optimizer\settings.py](file:///Logistics\route_optimizer\settings.py)
* **Functionality**:
* Manages app configurations. Loads environment variables from `env_var.env`.
* Defines `GOOGLE_MAPS_API_KEY`, `GOOGLE_MAPS_API_URL`, `USE_API_BY_DEFAULT`.
* API request settings: `MAX_RETRIES`, `BACKOFF_FACTOR`, `RETRY_DELAY_SECONDS`.
* `CACHE_EXPIRY_DAYS`.
* `TESTING` flag.
* **Important Points**:
* **Environment Variables**: Critical for API keys and sensitive data. `env_loader` assists local setup.
* **API Key Security**: `GOOGLE_MAPS_API_KEY` is vital; `.env` file must be in `.gitignore`.
* **Test Mode**: Allows different configurations for testing.
---
### Utils Files (`route_optimizer/utils/`)
#### 1. `env_loader.py` ([Logistics\route_optimizer\utils\env_loader.py](file:///Logistics\route_optimizer\utils\env_loader.py))
* **Functionality**:
* `load_env_from_file(file_path)`: Loads `KEY=VALUE` pairs from a file into `os.environ`.
* **Important Points**:
* **Local Development**: Useful for simulating production env vars locally.
* **Security**: The env file itself should be secure and not version-controlled.
#### 2. `helpers.py` ([Logistics\route_optimizer\utils\helpers.py](file:///Logistics\route_optimizer\utils\helpers.py))
* **Functionality**:
* Collection of miscellaneous utility functions: time conversions, Haversine calculation, route formatting, basic stats calculation, distance/time matrix creation, isolated node detection, safe JSON dumps, duration formatting.
* **Important Points**:
* **Redundancy**: Some functions might overlap with more specialized classes (e.g., Haversine vs. `DistanceMatrixBuilder`). Potential for refactoring.
* **Generic Utilities**: Small, focused helpers for general use.
---
### Other Project Files
* **`admin.py`** ([Logistics\route_optimizer\admin.py](file:///Logistics\route_optimizer\admin.py)): For Django admin interface. Currently empty.
* **`api/` directory**:
* **`serializers.py`** ([Logistics\route_optimizer\api\serializers.py](file:///Logistics\route_optimizer\api\serializers.py)): DRF serializers for API data validation and conversion.
* **`urls.py`** ([Logistics\route_optimizer\api\urls.py](file:///Logistics\route_optimizer\api\urls.py)): API URL patterns.
* **`views.py`** ([Logistics\route_optimizer\api\views.py](file:///Logistics\route_optimizer\api\views.py)): API views (`OptimizeRoutesView`, `RerouteView`) handling HTTP requests and responses.
* **`apps.py`** ([Logistics\route_optimizer\apps.py](file:///Logistics\route_optimizer\apps.py)): Django app configuration.
* **`migrations/`**: Django database migration files (e.g., for `DistanceMatrixCache`).
* **`models.py`** ([Logistics\route_optimizer\models.py](file:///Logistics\route_optimizer\models.py)): Dataclasses for `Vehicle` and `Delivery`; Django model for `DistanceMatrixCache`.
* **`tests/`**: Unit and integration tests.
* **`conftest.py`** ([Logistics\route_optimizer\tests\conftest.py](file:///Logistics\route_optimizer\tests\conftest.py)): Pytest configuration.
* **`test_settings.py`** ([Logistics\route_optimizer\tests\test_settings.py](file:///Logistics\route_optimizer\tests\test_settings.py)): Django settings for tests.
* **`views.py` (root)** ([Logistics\route_optimizer\views.py](file:///Logistics\route_optimizer\views.py)): Standard Django views file, currently empty.
---
This comprehensive overview should help in understanding the structure, functionality, and key considerations of the `route_optimizer` module.
Minor Improvements
removed potential inconsistencies, and possible bugs
## Route Optimizer Module - Pull Request Documentation
### Overview
The **Route Optimizer** module is a Django app designed to calculate optimal routes for a fleet of vehicles to perform deliveries or pickups for a set of locations. It is built to handle various real-world complexities, including:
* **Vehicle Constraints**: Considers vehicle capacities.
* **Time Constraints**: Can factor in time windows for deliveries and services.
* **Real-world Conditions**: Optionally incorporates traffic conditions.
* **Cost Optimization**: Aims to minimize operational costs.
* **Data Sources**: Can utilize local calculation methods (e.g., Haversine for distances) or integrate with external APIs (like Google Maps Distance Matrix API) for more accurate data.
* **Dynamic Adjustments**: Supports dynamic rerouting capabilities in response to real-time events such as traffic changes, service delays, or roadblocks.
The module is structured into several key areas: `core` for fundamental logic, `services` for orchestrating tasks, `api` for external communication, `utils` for helper functions, and standard Django components like `models`, `settings`, and `apps`.
---
## File-by-File Functionality Breakdown
### `route_optimizer/admin.py` ([Logistics\route_optimizer\admin.py](file:///Logistics\route_optimizer\admin.py))
* **Functionality**:
* This file is intended for registering Django models with the Django admin interface.
* Currently, it is empty, meaning no models from the `route_optimizer` app are explicitly registered for management via the admin panel.
* **To-Note**:
* If models like `DistanceMatrixCache` need to be inspected or managed via the Django admin, they would be registered here.
### `route_optimizer/api/serializers.py` ([Logistics\route_optimizer\api\serializers.py](file:///Logistics\route_optimizer\api\serializers.py))
* **Functionality**:
* Defines Django REST Framework (DRF) serializers for validating and converting data between API request/response formats and the internal Python/Django data structures (primarily DTOs from `core/types_1.py` and dataclasses from `models.py`).
* Key serializers include:
* `LocationSerializer`, `VehicleSerializer`, `DeliverySerializer`: For handling the primary input entities.
* `RouteOptimizationRequestSerializer`: Validates the input payload for the main optimization endpoint.
* `ReroutingRequestSerializer`: Validates the input payload for rerouting requests, including current routes and event-specific data (traffic, delays, roadblocks).
* `RouteSegmentSerializer`, `VehicleRouteSerializer`: Structure parts of the detailed route output.
* `StatisticsSerializer`, `ReroutingInfoSerializer`: For additional metadata in responses.
* `OptimizationResultSerializer`: A base or internal serializer for the `OptimizationResult` DTO.
* `RouteOptimizationResponseSerializer`: Defines the structure of the final successful response for optimization and rerouting endpoints, mapping fields from the internal `OptimizationResult` DTO (e.g., `detailed_routes` from DTO to `routes` in response).
* `TrafficDataSerializer`: Handles different formats of input traffic data.
* **Important Points**:
* **Data Validation**: Serializers are the first line of defense for ensuring incoming API data is correct and complete.
* **DTO Mapping**: Careful mapping between serializer fields and `OptimizationResult` DTO attributes is crucial, especially for nested structures like `detailed_routes`. Comments in the file indicate considerations for this mapping.
* **Clarity in API Response**: The `RouteOptimizationResponseSerializer` aims to provide a clear and usable structure for API clients, potentially renaming or restructuring fields from internal DTOs (e.g. DTO's `detailed_routes` field maps to `routes` in the response for client clarity).
* **Conditional Fields**: `ReroutingRequestSerializer` handles conditional fields based on `reroute_type` (e.g., `traffic_data` for 'traffic' type, `delayed_location_ids` for 'delay'). Its `validate` method contains logic for these conditions, though it's currently permissive for missing conditional data.
### `route_optimizer/api/urls.py` ([Logistics\route_optimizer\api\urls.py](file:///Logistics\route_optimizer\api\urls.py))
* **Functionality**:
* Defines the URL patterns for the `route_optimizer` API endpoints.
* Maps URLs to the corresponding API views in `api/views.py`.
* Includes:
* `/health/`: For a health check of the service ([`health_check`](file:///M:\Documents\B-Airways\Logistics\route_optimizer\api\views.py) view).
* `/optimize/`: For initiating new route optimizations ([`OptimizeRoutesView`](file:///M:\Documents\B-Airways\Logistics\route_optimizer\api\views.py)).
* `/reroute/`: For dynamic vehicle rerouting ([`RerouteView`](file:///M:\Documents\B-Airways\Logistics\route_optimizer\api\views.py)).
* **To-Note**:
* `app_name` is set to `'route_optimizer'`, allowing namespaced URL reversing.
* Uses `name` attributes for URL patterns (e.g., `optimize_routes_create`, `reroute_vehicles_update`, `health_check_get`) which match `operation_id` in Swagger documentation for easier API client generation and referencing.
### `route_optimizer/api/views.py` ([Logistics\route_optimizer\api\views.py](file:///Logistics\route_optimizer\api\views.py))
* **Functionality**:
* Contains the API view logic that handles HTTP requests, interacts with services, and formulates HTTP responses.
* `OptimizeRoutesView` (Class-based `APIView`):
* Handles `POST` requests to the `/optimize/` endpoint.
* Uses `RouteOptimizationRequestSerializer` to validate input.
* Instantiates DTOs (`Location`, `Vehicle`, `Delivery`) from validated data.
* Calls `OptimizationService.optimize_routes()` to perform the optimization.
* Maps the resulting `OptimizationResult` DTO to the `RouteOptimizationResponseSerializer` for the HTTP response.
* Includes `swagger_auto_schema` for API documentation generation.
* Handles conversion of different `traffic_data` input formats (`location_pairs` or `segments`) to the index-based format expected by `OptimizationService`.
* `RerouteView` (Class-based `APIView`):
* Handles `POST` requests to the `/reroute/` endpoint.
* Uses `ReroutingRequestSerializer` for input validation.
* Instantiates DTOs and converts `current_routes` JSON to `OptimizationResult` DTO using `OptimizationResult.from_dict()`.
* Calls appropriate methods on `ReroutingService` (e.g., `reroute_for_traffic`, `reroute_for_delay`, `reroute_for_roadblock`) based on `reroute_type`.
* Maps the `OptimizationResult` DTO from the rerouting service to `RouteOptimizationResponseSerializer`.
* `health_check` (Function-based view):
* Handles `GET` requests to `/health/`. Returns a simple "healthy" status.
* **Important Points**:
* **Error Handling**: Views include `try-except` blocks to catch exceptions from service layers or serialization, returning appropriate HTTP 500 or 400 responses.
* **DTO Conversion**: A key responsibility is converting serialized request data into the DTOs/dataclasses used by the service layer (e.g., `Location(**loc_data)`) and converting service DTO results back into serializable dictionaries for the response.
* **Logging**: Implements logging for errors and key events.
* **Status Code Logic**: The `OptimizeRoutesView` (and likely `RerouteView`) determines the HTTP response status code (200 OK or 400 Bad Request) based on the `status` field of the `OptimizationResult` DTO returned by the service layer.
### `route_optimizer/apps.py` ([Logistics\route_optimizer\apps.py](file:///Logistics\route_optimizer\apps.py))
* **Functionality**:
* Standard Django app configuration file.
* Defines the `RouteOptimizerConfig` class, inheriting from `AppConfig`.
* Sets `default_auto_field` and `name` for the app.
* Includes a `ready()` method, which can be used for app initialization tasks (e.g., importing signals), though it's currently empty.
* **To-Note**:
* `verbose_name` is set to 'Route Optimization Service'.
### `route_optimizer/core/constants.py` ([Logistics\route_optimizer\core\constants.py](file:///Logistics\route_optimizer\core\constants.py))
* **Functionality**:
* Defines various global constants used throughout the optimization process.
* Includes numerical scaling factors (e.g., `DISTANCE_SCALING_FACTOR`, `CAPACITY_SCALING_FACTOR`, `TIME_SCALING_FACTOR`) which are essential for OR-Tools to work correctly with integer arithmetic.
* Specifies safety bounds for distance and time values (e.g., `MAX_SAFE_DISTANCE`, `MAX_SAFE_TIME`) to prevent errors and handle edge cases.
* Default values for settings like delivery priority (`DEFAULT_DELIVERY_PRIORITY`, `PRIORITY_NORMAL`) are also defined here.
* **Important Points**:
* **Consistency is Key**: It is crucial that these constants, especially scaling factors and safety bounds like `MAX_SAFE_DISTANCE`, are used and understood consistently across all modules and their corresponding tests.
* **Scaling Factor Impact**: The choice of scaling factors directly influences the precision of calculations and the behavior of the OR-Tools solver. These may need tuning based on the typical range and magnitude of input data.
* Some constants like `MAX_ROUTE_DURATION_UNSCALED` and `COST_COEFFICIENT_FOR_LOAD_BALANCE` are defined directly within `ortools_optimizer.py` but are related in concept.
### `route_optimizer/core/dijkstra.py` ([Logistics\route_optimizer\core\dijkstra.py](file:///Logistics\route_optimizer\core\dijkstra.py))
* **Functionality**:
* Provides an implementation of Dijkstra's algorithm for finding the shortest paths in a graph.
* The `DijkstraPathFinder` class offers:
* `calculate_shortest_path(graph, start, end)`: Finds the shortest path and distance between a single pair of start and end nodes.
* `calculate_all_shortest_paths(graph, nodes)`: Calculates shortest paths between all specified pairs of nodes within the graph.
* `_validate_non_negative_weights(graph)`: An internal method to ensure that the graph does not contain negative edge weights, as standard Dijkstra's algorithm cannot handle them correctly.
* **Important Points**:
* **Negative Weights**: Explicitly checks for and raises a `ValueError` if negative edge weights are detected. If scenarios with negative weights (e.g., representing profits or bonuses) are required, an alternative algorithm like Bellman-Ford would be necessary.
* **Graph Representation**: Expects the input graph to be represented as a dictionary of dictionaries (an adjacency list format where `graph[node1][node2]` gives the weight of the edge from `node1` to `node2`).
* **Use Case**: This pathfinder is primarily used by the `PathAnnotator` service ([`Logistics\route_optimizer\services\path_annotation_service.py`](file:///Logistics\route_optimizer\services\path_annotation_service.py)) to generate detailed path segments for routes when external APIs are not being used for this purpose.
### `route_optimizer/core/distance_matrix.py` ([Logistics\route_optimizer\core\distance_matrix.py](file:///Logistics\route_optimizer\core\distance_matrix.py))
* **Functionality**:
* The `DistanceMatrixBuilder` class is responsible for creating and managing distance and time matrices, which are fundamental inputs for VRP solvers.
* **Calculation Methods**: Supports multiple methods for matrix calculation:
* Haversine formula for great-circle distances (local calculation).
* Euclidean distance (local calculation, less common for road travel).
* Google Maps Distance Matrix API for real-world road distances and travel times (requires API key).
* **Caching**: Implements caching for API-generated matrices using the `DistanceMatrixCache` Django model ([`Logistics\route_optimizer\models.py`](file:///Logistics\route_optimizer\models.py)) to reduce API calls and costs. Cache keys are generated based on input parameters.
* **API Interaction**:
* `create_distance_matrix_from_api(...)`: Handles requests to the Google Maps API, including retry logic (`_send_request_with_retry`) for transient network issues.
* `_process_api_response(...)`: Parses the JSON response from Google Maps API, converting distances to kilometers and times to minutes.
* **Matrix Manipulation**:
* `_sanitize_distance_matrix(...)`: Cleans matrices by replacing `NaN`, `inf`, and negative values with appropriate fallbacks (e.g., `MAX_SAFE_DISTANCE` or 0).
* `add_traffic_factors(...)` and `_apply_traffic_safely(...)`: Apply traffic adjustment factors to time matrices, with safety checks to cap extreme factors.
* `distance_matrix_to_graph(...)`: Converts a numerical distance matrix into a graph representation (dictionary of dictionaries) suitable for algorithms like Dijkstra's.
* **Important Points**:
* **API Key**: Relies on `GOOGLE_MAPS_API_KEY` from `route_optimizer.settings` ([`Logistics\route_optimizer\settings.py`](file:///Logistics\route_optimizer\settings.py)) when API usage is enabled.
* **Fallback Behavior**: If API calls fail or are not used, the system gracefully falls back to local calculation methods (typically Haversine).
* **Units**: Standardizes distances to kilometers and times to minutes.
* **Sanitization**: The `_sanitize_distance_matrix` method is critical for providing clean and numerically stable input to the VRP solvers, preventing errors from invalid matrix values.
* **Traffic Application**: The `_apply_traffic_safely` method includes bounds checking for traffic factors to prevent unrealistic travel times.
### `route_optimizer/core/ortools_optimizer.py` ([Logistics\route_optimizer\core\ortools_optimizer.py](file:///Logistics\route_optimizer\core\ortools_optimizer.py))
* **Functionality**:
* Encapsulates the logic for solving Vehicle Routing Problems (VRP) using Google's OR-Tools library.
* The `ORToolsVRPSolver` class provides methods to solve VRP variants:
* `solve(...)`: Solves the basic VRP, primarily considering vehicle capacities and minimizing total distance/cost.
* `solve_with_time_windows(...)`: Solves the VRP with Time Windows (VRPTW), respecting delivery time constraints for locations and vehicle operating hours.
* **Constraint Handling**: Manages various constraints including:
* Vehicle capacities (demand for deliveries).
* Vehicle start and end locations.
* Number of vehicles.
* Time windows for deliveries and vehicle service times (in `solve_with_time_windows`).
* **Callbacks**: Defines and registers essential callbacks for OR-Tools:
* Distance callback: Provides travel distance/cost between locations.
* Demand callback: Provides the demand (e.g., package volume/weight) for each delivery.
* Time callback (for VRPTW): Provides travel time, service time, and waiting time between locations.
* **Solution Processing**: Interprets the raw solution from OR-Tools and converts it into a standardized `OptimizationResult` DTO ([`Logistics\route_optimizer\core\types_1.py`](file:///Logistics\route_optimizer\core\types_1.py)). This includes extracting routes, total distance, assigned vehicles, and unassigned deliveries.
* **Load Balancing**: Includes logic to balance load (e.g., total route duration or distance) across vehicles using `SetGlobalSpanCostCoefficient`.
* **Important Points**:
* **Integer Scaling**: OR-Tools typically requires integer inputs for distances, times, and capacities. This solver uses scaling factors (e.g., `DISTANCE_SCALING_FACTOR`, `TIME_SCALING_FACTOR`, `CAPACITY_SCALING_FACTOR`) from `constants.py` ([`Logistics\route_optimizer\core\constants.py`](file:///Logistics\route_optimizer\core\constants.py)) to convert floating-point values to integers before passing them to the solver, and scales them back when interpreting the solution.
* **Depot Index**: The `depot_index` (index of the depot location in the distance matrix) is a fundamental parameter for the VRP setup.
* **Time Limits**: Solver behavior can be controlled by a `time_limit_seconds` parameter, preventing excessively long computation times.
* **Empty Problem Handling**: If no deliveries are provided, it generates simple depot-to-depot routes for each vehicle.
* **Cost Coefficients**: The `COST_COEFFICIENT_FOR_LOAD_BALANCE` and other internal OR-Tools cost settings can be tuned to influence solver priorities.
### `route_optimizer/core/types_1.py` ([Logistics\route_optimizer\core\types_1.py](file:///Logistics\route_optimizer\core\types_1.py))
* **Functionality**:
* Defines core Data Transfer Objects (DTOs) using Python's `dataclass` feature. These DTOs ensure a standardized and type-safe way to pass complex data structures between different parts of the application (services, core logic, API layers).
* Key DTOs include:
* `Location`: Represents a geographical point, including coordinates, depot status, time windows (start, end), and service time.
* `OptimizationResult`: The primary DTO for encapsulating the output of any optimization or rerouting process. It includes fields for status, routes (simple and detailed), total distance/cost, assigned vehicles, unassigned deliveries, and statistics. It has a crucial `from_dict` static method for reconstruction from dictionary data (e.g., from cache or API).
* `RouteSegment`: Details a specific segment of a route between two locations, including the path taken, distance, and estimated time.
* `DetailedRoute`: Provides a comprehensive description of a single vehicle's journey, including its ID, list of stops, segments, capacity utilization, and estimated arrival times.
* `ReroutingInfo`: Contains information specific to a rerouting operation, such as the reason for rerouting, traffic factors considered, or number of completed/remaining deliveries.
* `validate_optimization_result(result: Dict[str, Any]) -> bool`: A validation function to check the structural integrity and presence of key fields within an `OptimizationResult` (when represented as a dictionary).
* **Important Points**:
* **Standardization & Type Safety**: DTOs are crucial for maintaining consistency in data handling across different modules and for leveraging type hinting for better code quality and maintainability.
* **Data Integrity**: The `validate_optimization_result` function helps ensure that results produced by the optimization process adhere to an expected structure, which is important before caching or sending to an API.
* **Mutability**: Standard Python dataclasses are mutable by default. If immutability is desired for certain DTOs, `frozen=True` could be used.
* **Serialization/Deserialization**: While DTOs provide structure, actual serialization to/from JSON for API or caching is handled by DRF serializers ([`Logistics\route_optimizer\api\serializers.py`](file:///Logistics\route_optimizer\api\serializers.py)) or custom logic (like `OptimizationResult.from_dict`).
### `route_optimizer/migrations/0001_initial.py` ([Logistics\route_optimizer\migrations\0001_initial.py](file:///Logistics\route_optimizer\migrations\0001_initial.py))
* **Functionality**:
* This is the initial Django database migration file for the `route_optimizer` app.
* It defines the database schema for models created in this app, primarily the `DistanceMatrixCache` model.
* Includes operations to create the table for `DistanceMatrixCache` with its fields (`cache_key`, `matrix_data`, `location_ids`, `time_matrix_data`, `created_at`) and database indexes.
* **Important Points**:
* **Schema Definition**: This file is auto-generated by Django's `makemigrations` command based on `models.py`. It should not typically be edited manually.
* **Database Consistency**: Ensures that the database schema matches the model definitions in the code.
### `route_optimizer/models.py` ([Logistics\route_optimizer\models.py](file:///Logistics\route_optimizer\models.py))
* **Functionality**:
* Defines the data models for the `route_optimizer` application.
* **Dataclasses**:
* `Location`: (Defined in `core/types_1.py` but often referenced as a model concept). Represents physical locations with attributes like ID, coordinates, name, depot status, time windows, and service time.
* `Vehicle`: Dataclass representing a vehicle with attributes like ID, capacity, start/end location IDs, cost per km, fixed cost, and skills.
* `Delivery`: Dataclass representing a delivery task with attributes like ID, location ID, demand, priority, required skills, and pickup status.
* **Django Model**:
* `DistanceMatrixCache`: A Django model (`django.db.models.Model`) used to store cached distance and time matrices generated by the `DistanceMatrixBuilder` ([`Logistics\route_optimizer\core\distance_matrix.py`](file:///Logistics\route_optimizer\core\distance_matrix.py)). This helps reduce redundant calculations and API calls. Fields include `cache_key`, `matrix_data` (JSON serialized distance matrix), `location_ids` (JSON serialized), `time_matrix_data` (JSON serialized time matrix), and `created_at`. It includes database indexes on `cache_key` and `created_at` for efficient querying.
* **Important Points**:
* **Data Persistence**: `DistanceMatrixCache` is the only model in this file that directly maps to a database table for persistent storage.
* **Data Structures**: The dataclasses (`Vehicle`, `Delivery`) are used as structured data containers throughout the application, particularly for inputs to services and the VRP solver. They are not Django models and are not stored in the database directly unless serialized into other models.
* **Relationship with DTOs**: The dataclasses for Vehicle and Delivery are closely related to, and often instantiated from, data coming through serializers which might represent the `Location`, `Vehicle`, and `Delivery` DTOs/concepts from `core/types_1.py`.
### `route_optimizer/README.md` ([Logistics\route_optimizer\README.md](file:///Logistics\route_optimizer\README.md))
* **Functionality**:
* This file itself. It provides a comprehensive overview of the `route_optimizer` module, its purpose, core components, and a file-by-file breakdown of functionality and important considerations.
* **To-Note**:
* This document serves as the primary human-readable guide to understanding the module's architecture and how different parts interact. It should be kept up-to-date as the codebase evolves.
### `route_optimizer/services/depot_service.py` ([Logistics\route_optimizer\services\depot_service.py](file:///Logistics\route_optimizer\services\depot_service.py))
* **Functionality**:
* The `DepotService` class provides utility functions related to identifying and managing depot locations within a list of all provided locations.
* `get_nearest_depot(locations)`: Identifies a depot from the list of locations. Currently, its logic is simple: it returns the first location marked as `is_depot=True`. If no explicit depot is found, it defaults to returning the first location in the list.
* `find_depot_index(locations)`: Returns the numerical index of the depot location within the input list of locations. Similar to `get_nearest_depot`, it defaults to index 0 if no explicit depot is found or if the list is empty.
* **Important Points**:
* **Depot Assumption**: The current implementation assumes a single primary depot for routing problems or handles multiple depots by simply picking the first one encountered. More complex multi-depot VRP scenarios would require more sophisticated logic here or in the VRP solver setup.
* **Fallback Behavior**: The fallback to using the first location as a depot if none are explicitly marked is a crucial default behavior that ensures the VRP solver has a required start/end point.
### `route_optimizer/services/external_data_service.py` ([Logistics\route_optimizer\services\external_data_service.py](file:///Logistics\route_optimizer\services\external_data_service.py))
* **Functionality**:
* The `ExternalDataService` is designed to be responsible for fetching and processing external data that can affect route optimization, such as real-time traffic conditions, weather information, and road blockades.
* Methods like `get_traffic_data`, `get_weather_data`, and `get_roadblock_data` are defined.
* Includes logic for making API requests (`_make_api_request`) with retries and handling common request exceptions.
* Currently, if `use_mocks` is true (can be set during initialization) or if API keys are not provided/real API integrations are not fully implemented, the service provides mock data for these external factors (e.g., `_mock_traffic_data`, `_mock_weather_data`, `_mock_roadblock_data`).
* Provides a helper `combine_traffic_and_weather` to merge factors from different sources.
* **Important Points**:
* **Mock vs. Real Data**: For production use, the mock data generation would need to be replaced with actual integrations with relevant third-party APIs.
* **API Key Management**: If real APIs were used, this service would require proper API key management, likely sourcing keys from settings or environment variables (as seen with `traffic_api_key` and `weather_api_key` constructor parameters).
* **Fallback to Mock**: The service is designed to fall back to mock data if API keys are missing or API calls fail, ensuring some data is always available.
### `route_optimizer/services/optimization_service.py` ([Logistics\route_optimizer\services\optimization_service.py](file:///Logistics\route_optimizer\services\optimization_service.py))
* **Functionality**:
* The `OptimizationService` acts as the main orchestrator for the entire route optimization process. It ties together various components from the `core` and other `services` to generate an optimized route plan.
* **`optimize_routes(...)`**: This is the primary public method. Its responsibilities include:
* Input Validation (`_validate_inputs`): Checks for essential inputs like locations, vehicles, and deliveries, and validates their basic integrity.
* Caching: Implements caching for optimization results using Django's cache framework. It generates a cache key (`_generate_cache_key`) based on input parameters and retrieves/stores results to avoid re-computation.
* Distance Matrix Creation: Coordinates with `DistanceMatrixBuilder` to create distance/time matrices, deciding whether to use local calculations (e.g., Haversine) or external APIs based on the `use_api` flag and `USE_API_BY_DEFAULT` setting.
* Traffic Data Application: If `consider_traffic` is true and `traffic_data` is provided, it applies these factors to the matrix using `DistanceMatrixBuilder.add_traffic_factors`.
* Depot Identification: Uses `DepotService` to find the depot location and its index.
* VRP Solving: Invokes the `ORToolsVRPSolver` (`self.vrp_solver`) by calling either `solve()` or `solve_with_time_windows()` based on the `consider_time_windows` flag.
* Result Enrichment:
* `_add_detailed_paths(...)`: Populates the `detailed_routes` field in the `OptimizationResult`. If an external API (`use_api_flag` is true) was used for the main matrix, it attempts to use `TrafficService.create_road_graph` for path details; otherwise, it uses `PathAnnotator` with the computed distance matrix.
* `_add_summary_statistics(...)`: Calls `RouteStatsService.add_statistics` to calculate and add summary stats (costs, totals) to the result.
* **Initialization**: Allows injection of custom VRP solver and pathfinder instances, defaulting to `ORToolsVRPSolver` and `DijkstraPathFinder`.
* **Important Points**:
* **Central Orchestration**: This service is the main entry point for initiating an optimization and demonstrates the flow of data through various components.
* **Error Handling**: Includes a main `try-except` block in `optimize_routes` to catch general exceptions and return an `OptimizationResult` with `status='error'`. Specific validation errors also lead to an error status.
* **API Usage Control**: The decision to use external APIs (e.g., Google Maps) for distance calculations and detailed pathing is controlled by the `use_api` parameter and the `USE_API_BY_DEFAULT` setting from `route_optimizer.settings`.
* **DTO Consistency**: Ensures that the final output is a consistent `OptimizationResult` DTO, including handling cases where the underlying solver might return a dictionary that needs conversion via `OptimizationResult.from_dict()`.
* **Backward Compatibility**: Methods like `_add_detailed_paths` are designed to handle both dictionary-based results and `OptimizationResult` DTOs internally, likely for historical reasons or intermediate processing steps.
### `route_optimizer/services/path_annotation_service.py` ([Logistics\route_optimizer\services\path_annotation_service.py](file:///Logistics\route_optimizer\services\path_annotation_service.py))
* **Functionality**:
* The `PathAnnotator` class is responsible for enriching route optimization results with detailed segment-by-segment path information. This is typically used when external APIs (like Google Maps Directions) are not providing this level of detail directly.
* **`annotate(result, graph_or_matrix)`**: This is the main method. It iterates through the simple routes (lists of location IDs) in the `result` object. For each pair of consecutive stops in a route, it uses an injected `path_finder` (e.g., an instance of `DijkstraPathFinder` from `core/dijkstra.py`) to calculate the shortest path and distance between them. These path segments are then added to the `detailed_routes` section of the `result`.
* It can accept either a pre-computed graph (adjacency list with weights) or a distance matrix (which it can convert to a graph using `DistanceMatrixBuilder.distance_matrix_to_graph`) as input for the pathfinder.
* Handles both dictionary-based `result` objects and `OptimizationResult` DTOs for input and output.
* **`_add_summary_statistics(result, vehicles)`**: This helper method seems to ensure the basic structure of `detailed_routes` (list of dicts, each with `vehicle_id` and `stops`) and then calls `RouteStatsService.add_statistics`. This is a bit unusual as `RouteStatsService` is typically called by `OptimizationService` *after* path annotation. This might be a point for review or a specific internal use. (Update: The `_add_summary_statistics` method in `PathAnnotator` uses `RouteStatsService.add_statistics` from `route_optimizer.services.route_stats_service`. However, `OptimizationService` also has its own `_add_summary_statistics` method that does the same. The one in `PathAnnotator` might be for cases where `PathAnnotator` is used more standalone or to ensure stats are re-calculated after path details are added.)
* **Important Points**:
* **Dependency on Path Finder**: The quality and type of detailed paths depend heavily on the injected `path_finder`. For example, using `DijkstraPathFinder` will yield paths based on the provided graph/matrix, not necessarily real road networks unless the graph itself represents that.
* **Error Handling for Path Calculation**: If the `path_finder` fails to find a path between two stops (e.g., disconnected graph) or raises an exception, `annotate` logs the error and adds a placeholder segment with error information, ensuring the overall process doesn't halt.
* **Data Structure Handling**: It carefully manages whether it's working with a `dict` or an `OptimizationResult` DTO, ensuring `detailed_routes` are correctly initialized and populated.
### `route_optimizer/services/rerouting_service.py` ([Logistics\route_optimizer\services\rerouting_service.py](file:///Logistics\route_optimizer\services\rerouting_service.py))
* **Functionality**:
* The `ReroutingService` provides capabilities to dynamically adjust existing route plans in response to real-time events.
* It relies on an instance of `OptimizationService` to perform the actual re-optimization once the current state and new constraints are determined.
* **Key Methods**:
* `reroute_for_traffic(...)`: Adjusts routes based on new traffic information. `traffic_data` (mapping (from_idx, to_idx) to factors) is passed to the `OptimizationService`.
* `reroute_for_delay(...)`: Modifies routes due to service delays at specific locations. It updates the `service_time` for affected `Location` objects and triggers re-optimization, typically with `consider_time_windows=True`.
* `reroute_for_roadblock(...)`: Handles road blockages. It effectively makes the blocked segments impassable by setting their distance to infinity in a temporary distance matrix or by passing this information as `traffic_data` with infinite factors to `OptimizationService`.
* **Helper Methods**:
* `_get_remaining_deliveries(...)`: Filters the original list of deliveries to exclude those already marked as completed.
* `_update_vehicle_positions(...)`: A crucial but simplified method to estimate the current location of vehicles. It assumes a vehicle is at its next planned stop after its last completed delivery based on the `current_routes` plan. The `start_location_id` of the `Vehicle` objects are updated accordingly.
* **Important Points**:
* **State Management**: Accurate and up-to-date information about `completed_deliveries` and the true current positions/status of vehicles is critical for effective rerouting. The `_update_vehicle_positions` method is a simplified placeholder and might need more sophisticated logic (e.g., GPS tracking integration) in a real-world system.
* **Re-Optimization Cost**: Rerouting essentially triggers a new VRP solve, which can be computationally intensive. The scope and frequency of rerouting should be considered.
* **Input DTOs**: Methods expect `current_routes` as an `OptimizationResult` DTO, and other inputs as lists of `Location`, `Vehicle`, and `Delivery` DTOs/dataclasses.
* **ReroutingInfo**: Successful rerouting operations populate the `rerouting_info` field within the `statistics` dictionary of the returned `OptimizationResult` DTO, providing context about the reroute event.
### `route_optimizer/services/route_stats_service.py` ([Logistics\route_optimizer\services\route_stats_service.py](file:///Logistics\route_optimizer\services\route_stats_service.py))
* **Functionality**:
* The `RouteStatsService` is dedicated to calculating and adding various summary statistics to an optimization result.
* **`add_statistics(result, vehicles)`**: This static method is the main entry point.
* It calculates costs for each vehicle used (based on `fixed_cost` and `cost_per_km` from `Vehicle` objects and total distance from `detailed_routes` segments) and sums them up for `total_cost`.
* It aggregates overall statistics like total stops, total distance across all routes, number of vehicles used, and total deliveries assigned.
* It can handle input `result` as either a dictionary or an `OptimizationResult` DTO.
* If `detailed_routes` are not present in the `result` but simple `routes` (lists of location IDs) are, it will create a basic `detailed_routes` structure to enable some statistics calculation, though distances and costs might be incomplete in this case.
* **Important Points**:
* **Data Dependency for Costs**: Accurate cost calculation heavily depends on the `detailed_routes` field in the `result` being populated with `segments`, each having a `distance`. If segments are missing or distances are zero, variable costs will be underestimated.
* **Result Modification**: It modifies the input `result` object (dict or DTO) in place by adding or updating its `statistics` field and potentially `total_cost` and `detailed_routes`.
* **Vehicle Information**: Requires the list of `Vehicle` objects to access cost parameters (`fixed_cost`, `cost_per_km`).
### `route_optimizer/services/traffic_service.py` ([Logistics\route_optimizer\services\traffic_service.py](file:///Logistics\route_optimizer\services\traffic_service.py))
* **Functionality**:
* The `TrafficService` is intended to provide traffic-related information and utilities.
* **`apply_traffic_factors(matrix, traffic_data)`**: This static method is a wrapper around `DistanceMatrixBuilder.add_traffic_factors`, used to apply traffic factors to a given distance matrix.
* **`create_road_graph(locations)`**: Creates a road network graph from a list of `Location` objects.
* If an `api_key` is provided during `TrafficService` initialization (and thus available to `DistanceMatrixBuilder.create_distance_matrix_from_api`), this method attempts to use the Google Maps Distance Matrix API to get actual road distances and travel times to build the graph.
* If no API key is available or the API call fails, it falls back to using Haversine distances (via `_calculate_distance_haversine` and `DistanceMatrixBuilder.create_distance_matrix` with `use_haversine=True`) to construct the graph.
* The resulting graph is typically an adjacency list representation (dictionary of dictionaries) where edges store properties like distance and time.
* **Important Points**:
* **API Integration for Road Graph**: The `create_road_graph` method is a key integration point for leveraging external APIs to build a more realistic road network representation, which is crucial if `OptimizationService` is configured with `use_api=True` for detailed path generation via `_add_detailed_paths`.
* **Fallback Mechanisms**: Like other services interacting with external APIs, it has fallback mechanisms to ensure functionality even if API access is unavailable.
* **Haversine as Fallback**: `_calculate_distance_haversine` is a helper for fallback distance calculation.
### `route_optimizer/settings.py` ([Logistics\route_optimizer\settings.py](file:///Logistics\route_optimizer\settings.py))
* **Functionality**:
* Manages application-specific configurations for the `route_optimizer` module.
* Uses `load_env_from_file` (from `utils/env_loader.py`) to load environment variables from a file (e.g., `env_var.env` or `.env`), which is common for local development.
* Defines key settings such as:
* `GOOGLE_MAPS_API_KEY`: The API key for Google Maps services.
* `GOOGLE_MAPS_API_URL`: The base URL for the Google Maps Distance Matrix API.
* `USE_API_BY_DEFAULT`: A boolean flag determining whether external APIs should be used by default if not explicitly specified in service calls.
* API request parameters: `MAX_RETRIES`, `BACKOFF_FACTOR`, `RETRY_DELAY_SECONDS` for controlling retry behavior of external API calls.
* `CACHE_EXPIRY_DAYS`: Default expiry for cached items like distance matrices.
* `TESTING`: A flag, typically set by test configurations ([`Logistics\route_optimizer\tests\test_settings.py`](file:///Logistics\route_optimizer\tests\test_settings.py)), to alter behavior during tests (e.g., disable external calls, use dummy cache).
* `OPTIMIZATION_RESULT_CACHE_TIMEOUT`: Specific timeout for caching optimization results.
* **Important Points**:
* **Environment Variables**: Critical for managing sensitive data like API keys. The `.env` file used for this should be included in `.gitignore` and not committed to version control.
* **Centralized Configuration**: Provides a single place to manage how the application interacts with external services and its default operational parameters.
* **Test vs. Production Settings**: The `TESTING` flag allows for different configurations when running tests, which is essential for creating reliable and fast test suites (e.g., by mocking external dependencies or using in-memory caches).
### `route_optimizer/utils/env_loader.py` ([Logistics\route_optimizer\utils\env_loader.py](file:///Logistics\route_optimizer\utils\env_loader.py))
* **Functionality**:
* Provides a utility function `load_env_from_file(file_path)`.
* This function reads a specified file (typically a `.env` file) containing `KEY=VALUE` pairs, parses each line, and loads these pairs as environment variables into the current process's environment using `os.environ`.
* It skips empty lines and lines starting with `#` (comments).
* **Important Points**:
* **Local Development Aid**: Primarily used to simplify local development by allowing developers to set environment variables through a file instead of setting them system-wide or in shell profiles. This is particularly useful for API keys and other configuration that shouldn't be hardcoded.
* **File Existence and Errors**: Logs a warning if the specified file is not found and an error if issues occur during file parsing or setting environment variables.
* **Security**: The `.env` file itself, containing potentially sensitive information, must be kept secure and should always be listed in the `.gitignore` file to prevent accidental commitment to version control.
### `route_optimizer/utils/helpers.py` ([Logistics\route_optimizer\utils\helpers.py](file:///Logistics\route_optimizer\utils\helpers.py))
* **Functionality**:
* A collection of miscellaneous utility functions used across various parts of the `route_optimizer` module. These functions are generally small, focused, and provide common, reusable logic.
* Examples of functions include:
* Time conversions: `convert_minutes_to_time_str`, `convert_time_str_to_minutes`, `format_duration`.
* Haversine distance calculation (though `DistanceMatrixBuilder` also has this).
* Route formatting for display: `format_route_for_display`.
* Applying external factors to matrices: `apply_external_factors` (may have overlap with `DistanceMatrixBuilder`).
* Graph utilities: `detect_isolated_nodes`.
* Safe JSON serialization: `safe_json_dumps` to handle non-standard types like `datetime` or `numpy` arrays when converting objects to JSON strings.
* **Important Points**:
* **Redundancy Review**: Some functions in this file might have overlapping functionality with methods in more specialized classes (e.g., Haversine calculation in `DistanceMatrixBuilder`, traffic factor application). This could be an area for future refactoring to consolidate logic and improve maintainability by ensuring a single source of truth for certain operations.
* **General Purpose**: These helpers are intended for generic tasks that don't fit neatly into one of the main service or core logic classes.
* **JSON Serialization**: `safe_json_dumps` is particularly useful for robust logging or debugging when dealing with complex objects that might not be directly JSON serializable.
### `route_optimizer/views.py` (root level) ([Logistics\route_optimizer\views.py](file:///Logistics\route_optimizer\views.py))
* **Functionality**:
* This is the standard Django views file for the `route_optimizer` app, typically used for rendering HTML templates or handling web UI requests.
* Currently, it is empty, containing only the default `from django.shortcuts import render` and a comment `# Create your views here.`.
* **To-Note**:
* All API-related view logic is located in `route_optimizer/api/views.py`. This root `views.py` would be used if the app served any traditional Django web pages.
### `route_optimizer/__init__.py` ([Logistics\route_optimizer\__init__.py](file:///Logistics\route_optimizer\__init__.py))
* **Functionality**:
* Standard Python package initializer file for the `route_optimizer` directory.
* It makes the `route_optimizer` directory a Python package.
* Contains a docstring describing the module and a `__version__` attribute.
* **To-Note**:
* Can be used to control what symbols are exported when the package is imported with `from route_optimizer import *`, or to execute package-level initialization code, though it's minimal here.
Chamindu24
approved these changes
May 16, 2025
Contributor
|
Please resolve conflicts @L0rd008 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Route Optimizer Module Functionality
Overview
The Route Optimizer module is a Django application engineered to compute optimal routes for a fleet of vehicles tasked with deliveries and/or pickups across a designated set of locations. It is designed to handle a variety of real-world operational constraints and features, including:
The module is architecturally divided into several key directories:
core/: Contains fundamental algorithms, core data type definitions (DTOs), and essential constants.services/: Houses service classes that orchestrate complex business logic and workflows.api/: Manages external communication via RESTful APIs, including request handling, serialization, and response formulation.utils/: Provides miscellaneous helper functions and utilities.It also includes standard Django components like
models.pyfor data persistence (e.g., caching),settings.pyfor configurations, andapps.pyfor app-specific Django settings.Core Functionality & Modules
1. API Layer (
route_optimizer/api/)The API layer serves as the primary interface for external systems to interact with the route optimization capabilities.
views.py(Logistics\route_optimizer\api\views.py)OptimizeRoutesView(APIView):POSTrequests to the/optimize/endpoint.RouteOptimizationRequestSerializerto validate input data (locations, vehicles, deliveries, constraints).Location,Vehicle,Delivery).OptimizationService.optimize_routes()to perform the core optimization logic.OptimizationService.OptimizationResultDTO returned by the service into an HTTP response usingRouteOptimizationResponseSerializer.RerouteView(APIView):POSTrequests to the/reroute/endpoint.ReroutingRequestSerializerto validate input, which includes the current route plan and details about the rerouting event (traffic, delay, roadblock).current_routesinto anOptimizationResultDTO.ReroutingService(e.g.,reroute_for_traffic,reroute_for_delay,reroute_for_roadblock) based on thereroute_typespecified in the request.OptimizationResultDTO from the rerouting service into an HTTP response.health_check(request):GETrequests to/health/.{"status": "healthy"}to indicate the service is operational.Location(**loc_data).optimization_service.optimize_routes(...),rerouting_service.reroute_for_traffic(...).Response(serializer.data, status=...).@swagger_auto_schemafor automated API documentation generation (integrates withdrf-yasg).serializers.py(Logistics\route_optimizer\api\serializers.py)LocationSerializer,VehicleSerializer,DeliverySerializer: Define the expected structure and validation rules for individual location, vehicle, and delivery objects provided in requests.TrafficDataSerializer: Validates traffic data input, supporting formats like lists of location pairs with factors or dictionary of segments.RouteOptimizationRequestSerializer: Validates the entire payload for the route optimization endpoint, including lists of locations, vehicles, deliveries, and flags likeconsider_traffic,consider_time_windows.ReroutingRequestSerializer: Validates payloads for rerouting, including the current route plan, event type (reroute_type), and event-specific data (e.g.,traffic_data,delayed_location_ids,blocked_segments).RouteSegmentSerializer,VehicleRouteSerializer: Define the structure for detailed route segments and complete vehicle routes within the response.ReroutingInfoSerializer,StatisticsSerializer: Structure additional metadata and statistics included in responses.OptimizationResultSerializer: A base serializer for theOptimizationResultDTO, potentially used for internal representations or as a foundation. It includes avalidatemethod that callsvalidate_optimization_resultfromcore/types_1.py.RouteOptimizationResponseSerializer: Defines the final structure of successful responses from optimization and rerouting endpoints. It maps the internalOptimizationResultDTO fields to a client-friendly JSON structure (e.g.,OptimizationResult.detailed_routesmaps to theroutesfield in the JSON response).urls.py(Logistics\route_optimizer\api\urls.py)api/views.py.path('optimize/', OptimizeRoutesView.as_view(), name='optimize_routes_create')path('reroute/', RerouteView.as_view(), name='reroute_vehicles_update')path('health/', health_check, name='health_check_get')operation_idin Swagger documentation often matches these names.2. Core Logic (
route_optimizer/core/)This directory contains the heart of the optimization engine, including algorithms, data structures, and fundamental constants.
constants.py(Logistics\route_optimizer\core\constants.py)DISTANCE_SCALING_FACTOR,CAPACITY_SCALING_FACTOR,TIME_SCALING_FACTORare crucial for converting floating-point numbers to integers, as required by OR-Tools.MAX_SAFE_DISTANCE,MIN_SAFE_DISTANCE,MAX_SAFE_TIME,MIN_SAFE_TIMEdefine valid ranges for distance and time values, used in sanitization and validation.PRIORITY_NORMAL,PRIORITY_HIGH, and aDEFAULT_DELIVERY_PRIORITY.MAX_SAFE_DISTANCEis particularly important for matrix sanitization.types_1.py(Logistics\route_optimizer\core\types_1.py)dataclass. These ensure structured and type-safe data exchange within the application.Location: Represents a geographical point with attributes likeid,name,latitude,longitude,is_depot,time_window_start,time_window_end,service_time.OptimizationResult: The standard output format for all optimization and rerouting operations. Contains fields likestatus,routes(list of location ID lists),total_distance,total_cost,assigned_vehicles,unassigned_deliveries,detailed_routes(list ofDetailedRoute-like dictionaries), andstatistics. Includes a static methodfrom_dictfor creating an instance from a dictionary (e.g., when loading from cache).RouteSegment: Describes a single leg of a route, includingfrom_location,to_location,path(list of intermediate nodes),distance, andestimated_time.DetailedRoute: A comprehensive structure for a vehicle's entire route, includingvehicle_id,stops,segments,total_distance,total_time,capacity_utilization, andestimated_arrival_times.ReroutingInfo: Holds metadata about a rerouting event, such asreason,traffic_factorscount,completed_deliveriescount,delay_locationslist, andblocked_segmentslist.validate_optimization_result(result: Dict[str, Any]): Validates the structure of a dictionary representation of anOptimizationResult, ensuring required fields are present and have correct types. This is used by serializers.OptimizationResult.from_dict()is key for handling data from sources like caches or older dictionary-based outputs.distance_matrix.py(Logistics\route_optimizer\core\distance_matrix.py)DistanceMatrixBuilderclass for creating, caching, and manipulating distance and time matrices.create_distance_matrix(...): Main method to generate matrices.use_api=Trueand anapi_keyis provided, falling back to Haversine on failure.(distance_matrix_km, time_matrix_minutes_optional, location_ids_list).create_distance_matrix_from_api(...): Handles Google Maps API interaction, including request construction, sending (_send_request_with_retry), response parsing (_process_api_response), and caching viaDistanceMatrixCachemodel._sanitize_distance_matrix(matrix): Crucial for cleaning matrices: replacesNaNwithMAX_SAFE_DISTANCE,infwithMAX_SAFE_DISTANCE, negative values with 0, and caps excessively large values atMAX_SAFE_DISTANCE.add_traffic_factors(matrix, traffic_data): Applies traffic factors to a copy of the time matrix (or distance matrix if used as a proxy for time/cost). It calls_apply_traffic_safely._apply_traffic_safely(matrix, traffic_data): Applies traffic factors with bounds checking (e.g., factors < 1.0 are treated as 1.0, very large factors might be capped at amax_safe_factorwhich defaults to 5.0).distance_matrix_to_graph(matrix, location_ids): Converts a 2D numpy distance matrix into a dictionary-based graph representation (adjacency list).DistanceMatrixCachemodel) is vital for managing API costs and performance. Matrix sanitization (_sanitize_distance_matrix) ensures robust input for solvers. Fallback mechanisms (API to Haversine) enhance reliability.dijkstra.py(Logistics\route_optimizer\core\dijkstra.py)DijkstraPathFinderclass:_validate_non_negative_weights(graph): Ensures graph edges have non-negative weights, raisingValueErrorotherwise.calculate_shortest_path(graph, start, end): Finds the shortest path and distance between two nodes.calculate_all_shortest_paths(graph, nodes_subset): Calculates all-pairs shortest paths for a given subset of nodes within the larger graph.PathAnnotatorfor generating detailed path segments when not relying on external map APIs for this.ortools_optimizer.py(Logistics\route_optimizer\core\ortools_optimizer.py)ORToolsVRPSolverclass, which uses Google OR-Tools to solve Vehicle Routing Problems (VRP).solve(...): Solves the basic Capacitated VRP (CVRP).RoutingIndexManagerandRoutingModel.PATH_CHEAPEST_ARC,GUIDED_LOCAL_SEARCH,time_limit_seconds).routing.SolveWithParameters(search_parameters).OptimizationResultDTO, including routes, total distance (solver's objective), assigned vehicles, and unassigned deliveries.SetGlobalSpanCostCoefficienton a distance-based dimension.solve_with_time_windows(...): Solves VRP with Time Windows (VRPTW).solve()but adds a time dimension.OptimizationResultDTO, including estimated arrival times indetailed_routes.DISTANCE_SCALING_FACTOR,CAPACITY_SCALING_FACTOR, andTIME_SCALING_FACTORfromconstants.py.depot_indexis provided for all vehicles' start and end.MAX_ROUTE_DURATION_UNSCALED,MAX_ROUTE_DISTANCE_UNSCALED,COST_COEFFICIENT_FOR_LOAD_BALANCEare defined locally in this file and influence dimension capacities and load balancing.3. Service Layer (
route_optimizer/services/)The service layer orchestrates the core logic, acting as an intermediary between the API views and the core components.
optimization_service.py(Logistics\route_optimizer\services\optimization_service.py)OptimizationServiceclass:__init__(...): Initializes with an OR-Tools solver (ORToolsVRPSolver) and a pathfinder (DijkstraPathFinder), allowing for dependency injection. Defaults to creating new instances.optimize_routes(...): The main method._validate_inputsto check locations, vehicles, and deliveries._generate_cache_keyand attempts to retrieve results from Django's cache. If found, returns cachedOptimizationResult.DistanceMatrixBuilder.create_distance_matrix, deciding API vs. local calculation based onuse_apiflag and settings.DistanceMatrixBuilder._sanitize_distance_matrix.consider_trafficis true andtraffic_datais provided, applies it usingDistanceMatrixBuilder.add_traffic_factorsand re-sanitizes.DepotService.get_nearest_depotandDepotService.find_depot_index.self.vrp_solver.solve()orself.vrp_solver.solve_with_time_windows()based onconsider_time_windows.OptimizationResultDTO, converting from dict if necessary._add_detailed_paths): If optimization is successful:use_api_flagwas true, it attempts to useTrafficService.create_road_graph(which might use Google Maps API) to get a graph for path annotation.PathAnnotator(self.path_finder).annotate(...)to populatedetailed_routes._add_summary_statistics): CallsRouteStatsService.add_statistics.OptimizationResult(as a dict) in the cache.OptimizationResultDTO._validate_inputs(...): Checks for empty inputs and valid location coordinates/demands._generate_cache_key(...): Creates a cache key from input parameters.optimize_routesensures that exceptions lead to anOptimizationResultwithstatus='error'.rerouting_service.py(Logistics\route_optimizer\services\rerouting_service.py)ReroutingServiceclass:__init__(...): Takes an optionalOptimizationServiceinstance, or creates one._get_remaining_deliveries(...): Filters original deliveries against completed ones._update_vehicle_positions(...): A simplified method to estimate current vehicle locations based on their last completed delivery in thecurrent_routesplan. Updatesstart_location_idforVehicleobjects.reroute_for_traffic(...):self.optimization_service.optimize_routeswith the new state and providedtraffic_data.ReroutingInfoin the result's statistics.reroute_for_delay(...):Locationobjects based ondelayed_location_idsanddelay_minutes.self.optimization_service.optimize_routes, typically forcingconsider_time_windows=True.ReroutingInfo.reroute_for_roadblock(...):blocked_segmentstofloat('inf'). This is done by creating atraffic_data_for_roadblocksdictionary where blocked segments get an infinite factor.self.optimization_service.optimize_routeswithconsider_traffic=Trueand thetraffic_data_for_roadblocks.ReroutingInfo.current_routesinput and the logic in_update_vehicle_positions. Rerouting triggers a full re-optimization.path_annotation_service.py(Logistics\route_optimizer\services\path_annotation_service.py)PathAnnotatorclass:__init__(path_finder): Takes a pathfinding instance (e.g.,DijkstraPathFinder).annotate(result, graph_or_matrix):result(handlesdictorOptimizationResultDTO).result.detailed_routesis empty butresult.routes(simple list of location IDs) exists, it first populatesdetailed_routeswith basic stop information and assigned vehicle IDs.self.path_finder.calculate_shortest_pathusing the providedgraph_or_matrix(if it's a matrix, it's converted to a graph first usingDistanceMatrixBuilder.distance_matrix_to_graph).RouteSegmentlike dictionaries and adds them to thesegmentslist of the respectiveDetailedRoute._add_summary_statistics(result, vehicles): This internal helper seems to ensuredetailed_routeshave a basic structure and then callsRouteStatsService.add_statistics. (Note:OptimizationServicealso callsRouteStatsServiceafterPathAnnotator.annotate).path_finderand the inputgraph_or_matrix.route_stats_service.py(Logistics\route_optimizer\services\route_stats_service.py)RouteStatsServiceclass:add_statistics(result, vehicles)(static method):result(dict orOptimizationResultDTO) in place.detailed_routesis empty but simpleroutesexist, it creates basicdetailed_routesentries.vehicle_costs(fixed + variable based oncost_per_kmand segment distances fromdetailed_routes).total_costfor the entire solution.result.statistics['summary']liketotal_stops,total_distance,total_vehicles_used,total_deliveries_assigned.detailed_routeshaving segment distances.depot_service.py(Logistics\route_optimizer\services\depot_service.py)DepotServiceclass:get_nearest_depot(locations)(static method): Returns the firstLocationobject marked asis_depot=True. If none, returns the first location in the list.find_depot_index(locations)(static method): Returns the index of the depot. Defaults to 0 if no depot found.external_data_service.py(Logistics\route_optimizer\services\external_data_service.py)ExternalDataServiceclass:get_traffic_data,get_weather_data,get_roadblock_data._mock_traffic_data) if API keys are not set oruse_mocksis true._make_api_requesthelper with retry logic for actual API calls (if implemented).combine_traffic_and_weather: Merges factors from different sources.traffic_service.py(Logistics\route_optimizer\services\traffic_service.py)TrafficServiceclass:__init__(api_key=None): Can be initialized with an API key.apply_traffic_factors(matrix, traffic_data)(static): A wrapper forDistanceMatrixBuilder.add_traffic_factors.create_road_graph(locations):DistanceMatrixBuilder), it tries to use Google Maps API (viaDistanceMatrixBuilder.create_distance_matrix_from_api) to get distances/times for graph edges._calculate_distance_haversine).{'nodes': {}, 'edges': {from_node: {to_node: {'distance': km, 'time': secs}}}}.create_road_graphis key forOptimizationServicewhenuse_api=Trueto get detailed paths from a realistic road network.vrp_solver.py(Logistics\route_optimizer\services\vrp_solver.py)solve_with_time_windowsfunction. This function is very similar in its core logic (OR-Tools setup for VRPTW) to theORToolsVRPSolver.solve_with_time_windowsmethod found incore/ortools_optimizer.py.ORToolsVRPSolverclass incore/ortools_optimizer.pysuggests a potential area for refactoring to consolidate VRP solving logic into a single, authoritative place (likely theORToolsVRPSolverclass).OptimizationServiceflow. Review its current usage to determine if it can be deprecated or merged.4. Data Models & Migrations
models.py(Logistics\route_optimizer\models.py)Vehicle: Represents a vehicle withid,capacity,start_location_id,end_location_id,cost_per_km,fixed_cost,max_distance,max_stops,available,skills.Delivery: Represents a delivery/pickup task withid,location_id,demand,priority,required_skills,is_pickup.DistanceMatrixCache(models.Model): A Django model for caching results fromDistanceMatrixBuilder. Fields includecache_key(unique),matrix_data(JSON text),location_ids(JSON text),time_matrix_data(JSON text, nullable),created_at. Has indexes oncache_keyandcreated_at.migrations/0001_initial.py(Logistics\route_optimizer\migrations\0001_initial.py)DistanceMatrixCachetable in the database according to its model definition.5. Configuration & Utilities
settings.py(app-level) (Logistics\route_optimizer\settings.py)load_env_from_file(fromutils.env_loader).GOOGLE_MAPS_API_KEY,GOOGLE_MAPS_API_URL,USE_API_BY_DEFAULT.MAX_RETRIES,BACKOFF_FACTOR,RETRY_DELAY_SECONDS.CACHE_EXPIRY_DAYS,OPTIMIZATION_RESULT_CACHE_TIMEOUT.TESTINGflag, set dynamically based onsys.argvorsys.modules..envfile for secrets should be in.gitignore.utils/env_loader.py(Logistics\route_optimizer\utils\env_loader.py)load_env_from_file(file_path): Reads a.envfile, parsesKEY=VALUEpairs, and sets them as environment variables usingos.environ. Skips comments and empty lines..envfile itself must be kept secure.utils/helpers.py(Logistics\route_optimizer\utils\helpers.py)convert_minutes_to_time_str,convert_time_str_to_minutes,format_duration.format_route_for_display.apply_external_factors(potential overlap withDistanceMatrixBuilder).detect_isolated_nodes.safe_json_dumpsfor robust serialization of complex objects (handlesdatetime,numpytypes, etc.).6. Standard Django Files
admin.py(Logistics\route_optimizer\admin.py)DistanceMatrixCachecould be registered here for admin panel management.apps.py(Logistics\route_optimizer\apps.py)RouteOptimizerConfig.ready()method can be used for app initialization tasks (currently empty).views.py(root-level) (Logistics\route_optimizer\views.py)api/views.py.__init__.pyfilesroute_optimizer/,api/,core/,migrations/,services/,utils/.route_optimizer/__init__.pyalso defines__version__.README.md(Logistics\route_optimizer\README.md)7. Testing (
route_optimizer/tests/)While not part of the core runtime functionality, the
tests/directory is crucial for ensuring the module's reliability and correctness.api/,core/,services/,utils/).test_serializers.py,test_views.py: Test API layer components.test_dijkstra.py,test_distance_matrix.py,test_ortools_optimizer.py,test_types.py: Test core logic.test_optimization_service.py.test_models.py: Tests Django models and dataclass behavior.test_env_loader.py,test_helpers.py: Test utility functions.conftest.py(Logistics\route_optimizer\tests\conftest.py): Pytest configuration, sets up Django environment usingtest_settings.py.test_settings.py(Logistics\route_optimizer\tests\test_settings.py): Django settings specifically for tests (e.g., in-memory database, dummy API keys, disabled external calls, specificSECRET_KEY).__init__.py(Logistics\route_optimizer\tests_init_.py): Ensures test settings are loaded.