v3.0.0

What's Changed

Starting with this release, in addition to Docker Hub, images are now available through Amazon ECR Public Gallery and GitHub Container Registry.

This is a performance improvement release, focused on memory and CPU efficiency. The improvements introduce some backwards breaking changes. Details about changes needed during upgrade are covered in RELEASE_NOTES.

📝 With the release of v3.0.0, v2.x enters maintenance for support until April 1, 2026. We recommend migrating to Refinery 3.

💥 Breaking Changes 💥

• perf: streamline data handling by reducing the number of data transforms by @robbkidd in #1653
• fix: set DisableRedistribution default to true by @VinozzZ in #1657
• maint: separate metrics for otlp traffic based on transport protocol and incoming data format/signal by @VinozzZ in #1651
• fix: increment incoming/peer_router_span regardless of queue state by @VinozzZ in #1659
• maint: (BREAKING) remove CacheCapacity functionality by @VinozzZ in #1666
• maint: remove UpstreamBufferSize and PeerBufferSize from stress relief by @VinozzZ in #1672
• feat: allow deprecating an entire config group by @VinozzZ in #1692
• maint: remove deprecated config options by @VinozzZ in #1689
• maint: remove deprecated LegacyMetrics by @VinozzZ in #1691

💡 Enhancements

• perf: use noop span when tracing is disabled by @VinozzZ in #1583
• perf: only calculate span count and annotation type once by @VinozzZ in #1584
• perf: optimize trace key generation with hash-based deduplication by @VinozzZ in #1585
• perf: add lazy-loading MessagePack payload for events by @ianwilkes in #1586
• perf: use concrete type for Counter, Gauge, and Histogram metrics by @VinozzZ in #1590
• perf: move collector queue metrics to be outside of the collect loop by @VinozzZ in #1587
• feat: draft DirectTransmit to replace libhoney for trace data transmission by @ianwilkes in #1595
• perf: memoize fields that don't exist on a payload by @VinozzZ in #1601
• perf: fix encoded type of map keys by @ianwilkes in #1603
• feat: add message size limits to DirectTransmission by @ianwilkes in #1602
• perf: switch to an async pool for DirectTranmission batch sends, test changes by @ianwilkes in #1604
• perf: more scalable DirectTransmission batching by @ianwilkes in #1605
• feat: switch to DirectTransmission in refinery main by @ianwilkes in #1606
• perf: prefetch span attributes during ingest and sampling by @VinozzZ in #1607
• perf: promote all write-only metadata fields as first class fields in Payload struct by @VinozzZ in #1609
• perf: use a buffer pool for http body reads by @ianwilkes in #1610
• perf: remove dynamic string creation for metric names by @VinozzZ in #1615
• perf: reduce lock contention for internal metrics by @VinozzZ in #1617
• perf: memoize sampling fields in router by @VinozzZ in #1620
• perf: avoid dynamic string creation for sampling fields with no data by @VinozzZ in #1629
• perf: use protobuf to msgp translator for otlp/http protobuf request by @VinozzZ in #1632
• perf: avoid an extra copy for otlp protobuf payload during unmarshaling by @VinozzZ in #1633
• perf: remove metricsnamer to avoid dynamic string concatenation by @VinozzZ in #1626
• feat: allow file peers to be hot-reloaded by @kentquirk in #1636
• perf: use protobuf to msgp translator for otlp/http JSON request by @VinozzZ in #1641
• feat: use husky's optimized direct translation for GRPC calls by @ianwilkes in #1643
• perf: optimize sampling key field extraction in payload processing by @VinozzZ in #1645
• feat: implement retry for direct transmit by @VinozzZ in #1635
• perf: optimized JSON batch handling by @ianwilkes in #1644
• feat: config option deprecation by @VinozzZ in #1667
• feat(convert): support updating deprecated field using convert tool by @VinozzZ in #1674
• perf: use single zstd decoder by @VinozzZ in #1677

🐛 Fixes

• fix: fully initialize Payload object during unmarshaling by @VinozzZ in #1611
• fix: address race condition in managing locks for internal OTel metrics by @robbkidd in #1619
• perf: use read lock in Prometheus metrics Down operation by @VinozzZ in #1624
• fix: correct type for processing a span from a peer by @robbkidd in #1627
• fix: implement missing libhoney metrics in DirectTransmit by @VinozzZ in #1631
• fix: sampling key fields extraction for root-prefixed sampling fields by @VinozzZ in #1637
• fix: Ignore computed fields during sampling key extraction by @VinozzZ in #1640
• fix: error handling in postOTLPTrace by @VinozzZ in #1656
• fix: repair bad rule example by @kentquirk in #1660
• fix: set shutdown wait time to be 2x of BatchTimeout by @VinozzZ in #1663
• fix: make sure all transmit metrics have correct libhoney prefix by @VinozzZ in #1661
• fix: race condition between metrics startup and DirectTransmit registering metrics by @VinozzZ in #1665
• fix: dataset name should only be url encoded only once by @VinozzZ in #1681
• fix: wait for redis connection retry before marking Refinery as unheathy by @VinozzZ in #1685
• fix: increase upstream tranmission timeout to 30s by @VinozzZ in #1687
• fix: set default sizes for queues by @robbkidd in #1694
• fix: only try to track usage when enabled by @TylerHelmuth in #1699

🛠 Maintenance

• test: improve existing integration benchmark, add new test and benchmark for Router.batch() by @ianwilkes in #1580
• test: add benchmark test for collect loop by @VinozzZ in #1582
• perf: cleanup superfluous allocations during event unmarshal by @ianwilkes in #1592
• test: enable DryRun in collect loop benchmark test by @VinozzZ in #1594
• maint: remove metro hash and use wyhash instead by @VinozzZ in #1597
• test: add benchmark for transmission by @ianwilkes in #1599
• maint: remove unused LibhoneyMetricsWrapper by @VinozzZ in #1612
• maint: clean up payload code and testing, fix broken benchmark by @ianwilkes in #1613
• test: add BenchmarkRouterBatch/otlp by @ianwilkes in #1614
• ci: extend deadline for a wait in a flaky test by @robbkidd in #1623
• maint: upgrade dynsampler-go to 0.6.3 by @VinozzZ in #1625
• test: add OTLP to app test and benchmark by @ianwilkes in #1634
• ci: fix flaky tests by @robbkidd in #1639
• ci: fix flaky TestDirectTransmissionQueueLengthGauge by @VinozzZ in #1638
• maint: only use one reader per sendBatch in ...

v3.0.0-beta3

v3.0.0-beta3

v3.0.0-beta2

⚠️ Do not run with with trace redistribution enabled or with (experimental) trace locality set to distribute.

Aside from those two config contraindications, this build will accept 2.x config and rules YAMLs, so rolling back to a previous version should be straight forward. The upcoming official 3.0.0 release might require modifications to config inputs.

v2.9.7

What's Changed

💡 Enhancements

• feat: add support for REFINERY_HONEYCOMB_LOGGER_ADDITIONAL_FIELDS to honeycomb logger by @TylerHelmuth in #1600

🐛 Fixes

• fix: wait for goroutines to finish before shutting down by @VinozzZ in #1593
• fix: make TestStableMaxAlloc less flaky by @VinozzZ in #1596
• fix: send http status code 400 for empty event payload by @VinozzZ in #1618
• fix: use unique port number for integration test by @VinozzZ in #1621

🛠 Maintenance

• maint: use collect.Start() in collector_test.go by @VinozzZ in #1591

Full Changelog: v2.9.6...v2.9.7

v2.9.6

What's Changed

This release aims to improve user experience by adding new metrics and making default values more sensible.
It also includes several bug fixes and maintenance updates.

⚠️ Starting with this release: when using a file for peer management, do not include the current Refinery host in the list of peers. An entry for "self" in the list was a workaround for a bug fixed in this release.

See full details in the Release Notes.

💡 Enhancements

• feat: add incoming_router_otlp_events and incoming_router_batch_events metrics by @VinozzZ in #1566

🐛 Fixes

• fix: wait for all goroutines inside collector to finish during shutdown by @VinozzZ in #1589
• fix: use sensible default value for HealthCheckTimeout and MaxExpiredTraces by @VinozzZ in #1573
• fix: Docs about the meta fields were wrong; fix them. by @kentquirk in #1576
• fix: TraceKey used in dynamic sampler should not create a new key if no key fields are found by @VinozzZ in #1575
• fix: use value from GetCollectionConfigVal for HealthCheckTimeout in mock config by @VinozzZ in #1577
• fix: close redis pubsub connection on shutdown by @VinozzZ in #1578
• fix: TestWhichShard don't need to include self in peer list by @VinozzZ in #1571
• fix: don't reset SendBy if trace has already expired by @VinozzZ in #1570
• fix: use configured identifier for host address resolution in file peer by @VinozzZ in #1548
• fix: decision spans should not be processed through stress relief by @VinozzZ in #1544
• fix: maintain original TraceTimeout value on redistribution by @VinozzZ in #1543
• fix: panic caused by empty traces after redistribution by @VinozzZ in #1546
• fix: update refinery to report effective config on update by @TylerHelmuth in #1545
• fix: only execute Gosched in tests by @VinozzZ in #1549
• fix: Add valuetype to rules datatype by @MikeGoldsmith in #1562

🛠 Maintenance

• maint: update husky to v0.36.0 by @MikeGoldsmith in #1559
• maint: Add autogenerated comment to config templates by @MikeGoldsmith in #1564
• docs: update _router_otlp description by @JamieDanielson in #1565
• maint(deps): bump the minor-patch group across 1 directory with 17 updates by @dependabot in #1572

🤷 Other Changes

• maint: publish images to sippycup by @TylerHelmuth in #1557
• maint: Add stronger warning about TraceLocalityMode by @verythorough in #1568
• maint: use hpsf for strings by @TylerHelmuth in #1581
• maint: Bump dependencies by @kentquirk in #1579
• maint: prepare for 2.9.6 release by @VinozzZ in #1588

New Contributors

• @verythorough made their first contribution in #1568

Full Changelog: v2.9.5...v2.9.6

v2.9.5

This release includes minor improvements documentation and Refinery internal metric visibility. It also introduces two features listed below. The Embedded Environment Variables feature enables environment variable interpretation in Refinery configuration file. The OpAMP feature is intended for internal use only and is not supported or recommended for production.

See full details in the Release Notes.

What's Changed

💡 Enhancements

• feat: Support arbitrary envvar replacement in config strings by @kentquirk in #1517
• feat: add experimental support for opamp(internal use only) by @codeboten in #1528

🐛 Fixes

• fix: update trace_id field to meta.trace_id by @codeboten in #1520
• fix: ensure SendKeyMode priority by @codeboten in #1527
• fix: add env var support for APIHost in telemetry cfg by @VinozzZ in #1536
• fix: flush internal metrics on shutdown by @VinozzZ in #1534
• fix: improve trace cache doc and metric by @VinozzZ in #1535

🛠 Maintenance

• maint: Mark TraceLocalityMode as experimetnal by @MikeGoldsmith in #1522
• docs: add missing metrics and extra details by @JamieDanielson in #1515
• docs: some copyedits from docs by @JamieDanielson in #1526
• maint: record sampler key cardinality as metric by @VinozzZ in #1537
• maint(deps): bump golang.org/x/net from 0.37.0 to 0.38.0 by @dependabot in #1529
• maint(deps): bump github.com/golang/snappy from 0.0.4 to 1.0.0 by @dependabot in #1519
• maint(deps): bump the minor-patch group with 15 updates by @dependabot in #1518

Full Changelog: v2.9.4...v2.9.5

v2.9.4

What's Changed

💡 Enhancements

• feat: Record the number of bytes per event by @MikeGoldsmith in #1510

🛠 Maintenance

• maint(deps): bump the minor-patch group with 6 updates by @TylerHelmuth in #1499
• maint: bump deps by @TylerHelmuth in #1511

Full Changelog: v2.9.3...v2.9.4

v2.9.3

What's Changed

🐛 Fixes

• fix: only record attributes when otel tracing is enabled by @VinozzZ in #1480
• fix: error from logrus when sending a trace by @nic-littlepay in #1478
• fix: dropped_from_stress description by @mterhar in #1486
• fix: don't assume attribute type in oteltracing by @VinozzZ in #1483
• fix: register kept_from_stress metric by @VinozzZ in #1487

🛠 Maintenance

• maint(deps): bump the minor-patch group with 3 updates by @dependabot in #1479

New Contributors

• @nic-littlepay made their first contribution in #1478
• @mterhar made their first contribution in #1486

Full Changelog: v2.9.2...v2.9.3

v2.9.2

Honeycomb recently updated its event ingest limit to 1MB per event instead of the former 100K per event.
This release enables Refinery to conform to the same limit and now supports sending events up to 1MB.
It also improves Refinery's internal metrics by introducing a new metric, incoming_router_otlp, to track the amount of OTLP requests Refinery has received.

Fixes

fix: only load config and rules once (#1470) | Tyler Helmuth
fix: add incoming/peer_router_otlp metrics for otlp traffic (#1474) | Yingrong Zhao

Maintenance

maint(deps): bump the minor-patch group across 1 directory with 14 updates (#1475) | dependabot
feat: update libhoney to 1.25.0 (#1473) | Yingrong Zhao
maint: bump golang.org/x/net to v0.33.0 (#1466) | Tyler Helmuth

v2.9.1

This is a bug fix release.

Fixes

• register OTEL metrics if it's missing (#1456) | Yingrong Zhao

Maintenance

• call out cache changes and add missing changelog entry (#1454) | Yingrong Zhao
• update Husky to 0.34.0 (#1459) | Mike Goldsmith