Skip to content

⚡️ Speed up LocalAggregator.to_json() by 38% in sentry_sdk/metrics.py #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
codeflash-ai wants to merge 1 commit into
base: master
Choose a base branch
from
Open

⚡️ Speed up LocalAggregator.to_json() by 38% in sentry_sdk/metrics.py #4

codeflash-ai wants to merge 1 commit into from

Conversation

@codeflash-ai
Copy link

@codeflash-ai codeflash-ai bot commented Jun 14, 2024

📄 LocalAggregator.to_json() in sentry_sdk/metrics.py

📈 Performance improved by 38% (0.38x faster)

⏱️ Runtime went down from 1.80 microseconds to 1.30 microsecond

Explanation and details

Certainly! Below is an optimized version of the provided Python program. The optimizations involved avoiding redundant dictionary lookups and using efficient ways to handle existing values.

Explanation of Optimizations.

  1. Dictionary Lookup Optimization in _tags_to_dict.

    • Changed rv.get(tag_name) to a direct membership check if tag_name in rv to reduce redundant lookups and improve readability.

  2. Predefine measurements in to_json.

    • Assigned self._measurements[export_key, tags] to measurements variable to avoid multiple dictionary lookups and make the code more readable.

These changes ensure that the code remains logically consistent with the original while improving the runtime performance by reducing redundant operations.

Correctness verification

The new optimized code was tested for correctness. The results are listed below.

🔘 (none found) − ⚙️ Existing Unit Tests

✅ 1 Passed − 🌀 Generated Regression Tests

(click to show generated tests) 
# imports
import pytest  # used for our unit tests


# function to test
def _tags_to_dict(tags):
    # type: (MetricTagsInternal) -> Dict[str, Any]
    rv = {}  # type: Dict[str, Any]
    for tag_name, tag_value in tags:
        old_value = rv.get(tag_name)
        if old_value is not None:
            if isinstance(old_value, list):
                old_value.append(tag_value)
            else:
                rv[tag_name] = [old_value, tag_value]
        else:
            rv[tag_name] = tag_value
    return rv
from sentry_sdk.metrics import LocalAggregator


# unit tests
def test_single_measurement_single_tag():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "value1")]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": "value1"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_single_measurement_multiple_tags():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "value1"), ("tag2", "value2")]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": "value1", "tag2": "value2"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_multiple_measurements_different_metrics():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric1", [("tag1", "value1")]): (1.0, 2.0, 1, 1.5),
        ("metric2", [("tag2", "value2")]): (2.0, 3.0, 1, 2.5)
    }
    expected = {
        "metric1": [
            {
                "tags": {"tag1": "value1"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ],
        "metric2": [
            {
                "tags": {"tag2": "value2"},
                "min": 2.0,
                "max": 3.0,
                "count": 1,
                "sum": 2.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_multiple_measurements_same_metric_different_tags():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric1", [("tag1", "value1")]): (1.0, 2.0, 1, 1.5),
        ("metric1", [("tag2", "value2")]): (2.0, 3.0, 1, 2.5)
    }
    expected = {
        "metric1": [
            {
                "tags": {"tag1": "value1"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            },
            {
                "tags": {"tag2": "value2"},
                "min": 2.0,
                "max": 3.0,
                "count": 1,
                "sum": 2.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_empty_measurements():
    aggregator = LocalAggregator()
    aggregator._measurements = {}
    expected = {}
    assert aggregator.to_json() == expected

def test_single_tag_multiple_values():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "value1"), ("tag1", "value2")]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": ["value1", "value2"]},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_tags_with_empty_string_values():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "")]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": ""},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_tags_with_none_values():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", None)]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": None},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_large_number_of_measurements():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric{}".format(i), [("tag{}".format(j), "value{}".format(j)) for j in range(10)]): (i, i+1, i, i+0.5) 
        for i in range(1000)
    }
    result = aggregator.to_json()
    assert len(result) == 1000
    for i in range(1000):
        assert "metric{}".format(i) in result
        assert len(result["metric{}".format(i)]) == 1

def test_large_number_of_tags_per_measurement():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric1", [("tag{}".format(i), "value{}".format(i)) for i in range(1000)]): (1.0, 2.0, 1, 1.5)
    }
    expected_tags = {"tag{}".format(i): "value{}".format(i) for i in range(1000)}
    result = aggregator.to_json()
    assert "metric1" in result
    assert len(result["metric1"]) == 1
    assert result["metric1"][0]["tags"] == expected_tags

def test_nested_tags_with_multiple_values():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "value1"), ("tag1", "value2"), ("tag2", "value3")]): (1.0, 2.0, 1, 1.5)}
    expected = {
        "metric1": [
            {
                "tags": {"tag1": ["value1", "value2"], "tag2": "value3"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_multiple_metrics_with_overlapping_tags():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric1", [("tag1", "value1")]): (1.0, 2.0, 1, 1.5),
        ("metric2", [("tag1", "value1"), ("tag2", "value2")]): (2.0, 3.0, 1, 2.5)
    }
    expected = {
        "metric1": [
            {
                "tags": {"tag1": "value1"},
                "min": 1.0,
                "max": 2.0,
                "count": 1,
                "sum": 1.5,
            }
        ],
        "metric2": [
            {
                "tags": {"tag1": "value1", "tag2": "value2"},
                "min": 2.0,
                "max": 3.0,
                "count": 1,
                "sum": 2.5,
            }
        ]
    }
    assert aggregator.to_json() == expected

def test_non_iterable_tags():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", "invalid_tags"): (1.0, 2.0, 1, 1.5)}
    with pytest.raises(TypeError):
        aggregator.to_json()

def test_non_tuple_measurement_values():
    aggregator = LocalAggregator()
    aggregator._measurements = {("metric1", [("tag1", "value1")]): "invalid_measurement_values"}
    with pytest.raises(ValueError):
        aggregator.to_json()

def test_stress_test_with_maximum_load():
    aggregator = LocalAggregator()
    aggregator._measurements = {
        ("metric{}".format(i), [("tag{}".format(j), "value{}".format(j)) for j in range(100)]): (i, i+1, i, i+0.5) 
        for i in range(10000)
    }
    result = aggregator.to_json()
    assert len(result) == 10000
    for i in range(10000):
        assert "metric{}".format(i) in result
        assert len(result["metric{}".format(i)]) == 1

🔘 (none found) − ⏪ Replay Tests

@codeflash-ai
⚡️ Speed up LocalAggregator.to_json() by 38%
8c641b1 
Certainly! Below is an optimized version of the provided Python program. The optimizations involved avoiding redundant dictionary lookups and using efficient ways to handle existing values.



### Explanation of Optimizations.

1. **Dictionary Lookup Optimization in `_tags_to_dict`**.
    - Changed `rv.get(tag_name)` to a direct membership check `if tag_name in rv` to reduce redundant lookups and improve readability.
  
2. **Predefine `measurements` in `to_json`**.
   - Assigned `self._measurements[export_key, tags]` to `measurements` variable to avoid multiple dictionary lookups and make the code more readable. 

These changes ensure that the code remains logically consistent with the original while improving the runtime performance by reducing redundant operations.
@codeflash-ai codeflash-ai bot added the ⚡️ codeflash Optimization PR opened by Codeflash AI label Jun 14, 2024
@codeflash-ai codeflash-ai bot requested a review from misrasaurabh1 June 14, 2024 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@misrasaurabh1 misrasaurabh1 Awaiting requested review from misrasaurabh1

Assignees

No one assigned

Labels

⚡️ codeflash Optimization PR opened by Codeflash AI

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

0 participants