feat(profiling): Route raw profiles through objectstore service#6025
Conversation
Extract StoreRawProfile into the objectstore service and replace inline raw profile blob with objectstore key reference in the Kafka message. Co-Authored-By: Claude <noreply@anthropic.com>
markushi
changed the base branch from
master
to
feat/markushi/perfetto-profiling-support-pipeline
Align the objectstore usecase identifier with the naming convention so it can be easily distinguished from a future profiles_v2 usecase. Co-Authored-By: Claude <noreply@anthropic.com>
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Add try_send_to_objectstore to StoreHandle, which returns the message when the objectstore service is not configured. Profile chunks fall back to sending the StoreProfileChunk directly to Store, preventing data loss. Also log session creation errors matching the pattern in other objectstore handlers. Co-Authored-By: Claude <noreply@anthropic.com>
The rename in 3a83bc5 only updated MessageKind::as_str() but missed the Usecase::new() call, causing raw profiles to be stored under the generic "profiles" namespace instead of "profiles_raw". Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| let trace_attachments = Usecase::new("trace_attachments") | ||
| .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION)); | ||
| let profiles = Usecase::new("profiles") | ||
| .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION)); |
There was a problem hiding this comment.
Note that DEFAULT_ATTACHMENT_RETENTION means event attachments here. Even if that happens to be the correct retention you actually want, you likely want to use a different constant here to not mix things up.
Here you likely want to use the profiles retention instead.
There was a problem hiding this comment.
Good point, I was wondering about this as well. Some quick research showed me that the monolith builds a project config, which then gets propagate to relay. And that config is coming from getsentry. At this point I feel like pulling on a thread a bit 😅 but I guess we need this, maybe @jjbayer could chime in here as well, before I start adding support for this.
We would need to
- [getsentry] Propagate the config based on the plan, looks like this already happens automatically as it iterates over all data categories
- [sentry] Extend the RETENTIONS_CONFIG_MAPPING to also include
DataCategory.PROFILE_DURATIONandDataCategory.PROFILE_DURATION_UI, so those get propagated to relay - [relay] Extend RetentionsConfig and use that config here
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 907aa48. Configure here.
| let trace_attachments = Usecase::new("trace_attachments") | ||
| .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION)); | ||
| let profiles = Usecase::new("profiles_raw") | ||
| .with_expiration_policy(ExpirationPolicy::TimeToLive(DEFAULT_ATTACHMENT_RETENTION)); |
There was a problem hiding this comment.
Wrong retention constant used for profile objectstore uploads
Medium Severity
The profiles usecase is initialized with DEFAULT_ATTACHMENT_RETENTION (30 days), which is the retention for event attachments. Profiles typically use event retention (90 days via DEFAULT_EVENT_RETENTION). While each upload call sets a per-request expiration policy via upload_bytes, the Usecase-level default acts as a fallback. If the per-request policy is ever not applied (e.g. retention_days * 24 overflows u16 via checked_mul returning None), stored raw profiles would be garbage-collected after 30 days instead of the correct profile retention, causing silent data loss. This was also flagged by a reviewer in the PR discussion.
Reviewed by Cursor Bugbot for commit 907aa48. Configure here.
2 participants
Summary
StoreRawProfileinto the objectstore service to upload raw profile blobs (e.g. Perfetto traces) before forwarding to KafkaStoreProfileChunkto the Store service, even if the upload fails, ensuring the Kafka message is producedTest plan
pytest tests/integration/test_profile_chunks_perfetto.py🤖 Generated with Claude Code