Skip to content

feat: Implement historical pricing for message cost recalculation #813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

amalcaraz
Copy link
Collaborator

Summary

Implements historical pricing support for message cost recalculation, ensuring costs are calculated using the pricing models that were active when messages were originally created.

Key Changes

🔧 New Pricing Timeline System

  • Created pricing_utils.py with reusable pricing model utilities
  • Implemented get_pricing_timeline() to track pricing changes over time
  • Added proper cumulative merging of pricing aggregate elements

⚡ Enhanced Cost Recalculation

  • Updated /api/v0/price/recalculate endpoint to use historical pricing
  • Messages processed chronologically (oldest to newest) with correct pricing timeline
  • Maintains pricing model state efficiently during batch processing

🔄 Migration Refactoring

  • Refactored migration 0033_1c06d0ade60c to use reusable build_default_pricing_model()
  • Removed hardcoded pricing dictionary in favor of DEFAULT_PRICE_AGGREGATE constant
  • Improved maintainability and consistency

🧪 Comprehensive Testing

  • Added 60+ test cases covering timeline functionality, API endpoints, and migration compatibility
  • Tests ensure historical accuracy and proper error handling
  • Integration tests verify end-to-end pricing application

API Endpoints

  • POST /api/v0/price/recalculate - Recalculate all message costs with historical pricing
  • POST /api/v0/price/{item_hash}/recalculate - Recalculate specific message costs

Benefits

✅ Historical Accuracy: Costs calculated using pricing active when messages were created✅ Performance: Efficient timeline processing without repeated DB queries✅ Maintainability:
Reusable pricing utilities reduce code duplication✅ Robustness: Comprehensive error handling and extensive test coverage

@amalcaraz amalcaraz requested a review from nesitor June 24, 2025 14:59
@amalcaraz amalcaraz self-assigned this Jun 24, 2025
@amalcaraz amalcaraz added the enhancement New feature or request label Jun 24, 2025
),
}


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we shouldn't have to modify a migration already executed on the rest of the CCNs to prevent unconsistencies on the database.

Comment on lines +176 to +186
"""Force recalculation of message costs in chronological order with historical pricing.

This endpoint will:
1. Get all messages that need cost recalculation (if item_hash provided, just that message)
2. Get the pricing timeline to track price changes over time
3. Sort messages chronologically (oldest first)
4. For each message, use the pricing model that was active when the message was created
5. Delete existing cost entries and recalculate with historical pricing
6. Store the new cost calculations
"""

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we commented, I think that we should to add a kind of temporary token authentication based to prevent that endpoint to be used by people to DDOS the CCN as the endpoints takes time to re-process everything again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants