Support tracing for async producing, batch sync consuming, and batch async consuming in rocketMQ-client-java-5.x-plugin by CzyerChen · Pull Request #665 · apache/skywalking-java

CzyerChen · 2024-01-19T03:05:03Z

Enhance an agent plugin to support RocketMQ

Add a test case for the new plugin, refer to the doc

Based on rocketMQ-client-java-5.x-plugin, add interceptors to tracing async producing, batch sync consuming, and batch async consuming cases. Batch consuming messages can refer to the producer. Async producing can record messageId.

If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes [Feature] add Support for RocketMQ newAPI rocketmq-client-java skywalking#11752.
Update the CHANGES log.

…async consuming in rocketMQ-client-java-5.x-plugin

wu-sheng · 2024-01-19T03:09:58Z

Could you share how batch consuming works? Is UI working as expected to see multiple relative trace ID from downstream?

CzyerChen · 2024-01-19T05:09:20Z

Could you share how batch consuming works? Is UI working as expected to see multiple relative trace ID from downstream?

As the testcase:

1.api

2.produce 1 normal message

3.PushConsumer consumes 1 message by listener

4.5.produce 2 normal messages async

6.SimpleConsumer consumes 2 messages by triggered manually

7.8.produce 2 normal messages

9.SimpleConsumer consumes 2 messages async by triggered manually

4 segments share 1 traceId

wu-sheng · 2024-01-19T12:58:00Z

So, from reading your graph, the same number of spans are going to be created based on the number of messages in one bulk. Am I right?

Q, what will happen if two messages are produced from different contexts (two trace IDs), and consumed in one bulk from one client, is this possible?

wu-sheng · 2024-01-19T12:59:24Z

BTW, at here, you need to use the whole URL, rather than issue ID.

- [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes https://github.com/apache/skywalking/issues/11752.

CzyerChen · 2024-01-19T13:58:14Z

So, from reading your graph, the same number of spans are going to be created based on the number of messages in one bulk. Am I right?

Q, what will happen if two messages are produced from different contexts (two trace IDs), and consumed in one bulk from one client, is this possible?

Testcase:
ProjectA:

produce 1 message
produce 1 message

ProjectB:

consumer 2 message in a batch

Then, the graph like this:

1 producer can be referred, the other can not.

The graphQL data is here:

{
    "data": {
        "trace": {
            "spans": [
                {
                    "traceId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540001",
                    "segmentId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540000",
                    "spanId": 0,
                    "parentSpanId": -1,
                    "refs": [],
                    "serviceCode": "rmq5-grpc-A-scenario",
                    "serviceInstanceName": "3d4a5d93c2af47838948691e07c502e6@198.18.0.1",
                    "startTime": 1705671679055,
                    "endTime": 1705671679135,
                    "endpointName": "GET:/case/rocketmq-5-grpc-scenario",
                    "type": "Entry",
                    "peer": "",
                    "component": "SpringMVC",
                    "isError": false,
                    "layer": "Http",
                    "tags": [
                        {
                            "key": "url",
                            "value": "http://127.0.0.1:8080/rocketmq-5-grpc-scenario/case/rocketmq-5-grpc-scenario"
                        },
                        {
                            "key": "http.method",
                            "value": "GET"
                        },
                        {
                            "key": "http.status_code",
                            "value": "200"
                        }
                    ],
                    "logs": [],
                    "attachedEvents": []
                },
                {
                    "traceId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540001",
                    "segmentId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540000",
                    "spanId": 1,
                    "parentSpanId": 0,
                    "refs": [],
                    "serviceCode": "rmq5-grpc-A-scenario",
                    "serviceInstanceName": "3d4a5d93c2af47838948691e07c502e6@198.18.0.1",
                    "startTime": 1705671679057,
                    "endTime": 1705671679133,
                    "endpointName": "RocketMQ/ConsumerAsyncTopicTest/Producer",
                    "type": "Exit",
                    "peer": "123.56.194.74:8081",
                    "component": "rocketMQ-producer",
                    "isError": false,
                    "layer": "MQ",
                    "tags": [
                        {
                            "key": "mq.broker",
                            "value": "123.56.194.74:8081"
                        },
                        {
                            "key": "mq.topic",
                            "value": "ConsumerAsyncTopicTest"
                        },
                        {
                            "key": "mq.message.keys",
                            "value": "e8029111-fa26-4738-a59f-2d4dab0b2ffe"
                        },
                        {
                            "key": "mq.message.tags",
                            "value": "Tag:async:consumer"
                        },
                        {
                            "key": "mq.message.id",
                            "value": "01ACDE480011221E0005BC15FF00000003"
                        }
                    ],
                    "logs": [],
                    "attachedEvents": []
                },
                {
                    "traceId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540001",
                    "segmentId": "1c116740e8dd41f9af3577a03c969a4b.53.17056716950020000",
                    "spanId": 0,
                    "parentSpanId": -1,
                    "refs": [
                        {
                            "traceId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540001",
                            "parentSegmentId": "d013fdb11b56417a9997db9d56722ce2.57.17056716790540000",
                            "parentSpanId": 1,
                            "type": "CROSS_PROCESS"
                        },
                        {
                            "traceId": "d013fdb11b56417a9997db9d56722ce2.55.17056716721260001",
                            "parentSegmentId": "d013fdb11b56417a9997db9d56722ce2.55.17056716721250000",
                            "parentSpanId": 1,
                            "type": "CROSS_PROCESS"
                        }
                    ],
                    "serviceCode": "rmq5-grpc-B-scenario",
                    "serviceInstanceName": "88429d7fb3434b928dc39f41c0b7c4c6@198.18.0.1",
                    "startTime": 1705671695004,
                    "endTime": 1705671696483,
                    "endpointName": "RocketMQ/ConsumerAsyncTopicTest/Consumer",
                    "type": "Entry",
                    "peer": "123.56.194.74:8081",
                    "component": "rocketMQ-consumer",
                    "isError": false,
                    "layer": "MQ",
                    "tags": [
                        {
                            "key": "mq.topic",
                            "value": "ConsumerAsyncTopicTest"
                        },
                        {
                            "key": "mq.broker",
                            "value": "123.56.194.74:8081"
                        },
                        {
                            "key": "transmission.latency",
                            "value": "17345"
                        },
                        {
                            "key": "transmission.latency",
                            "value": "24263"
                        },
                        {
                            "key": "http.status_code",
                            "value": "200"
                        }
                    ],
                    "logs": [],
                    "attachedEvents": []
                }
            ]
        }
    }
}

wu-sheng · 2024-01-19T14:06:20Z

What does the VNode mean in this case?

CzyerChen · 2024-01-19T14:17:14Z

What does the VNode mean in this case?

The VNode is another ref.

wu-sheng · 2024-01-19T14:19:25Z

OK, I will work with @Fine0830 about UI part, this should not be your concern. Thanks for your explanation.

...ava/org/apache/skywalking/apm/plugin/rocketMQ/client/java/v5/MessageListenerInterceptor.java

...va/org/apache/skywalking/apm/plugin/rocketMQ/client/java/v5/MessageSendAsyncInterceptor.java

...apache/skywalking/apm/plugin/rocketMQ/client/java/v5/SimpleConsumerImplAsyncInterceptor.java

… code & log when error occurred & update test scenario

apm-sniffer/config/agent.config

...org/apache/skywalking/apm/plugin/rocketMQ/client/java/v5/RocketMqClientJavaPluginConfig.java

…cing and consuming

Support tracing for async producing, batch sync consuming, and batch …

efd1e4a

…async consuming in rocketMQ-client-java-5.x-plugin

wu-sheng added this to the 9.2.0 milestone Jan 19, 2024

wu-sheng added the feature label Jan 19, 2024

add 2 tags in expectedData.yaml

c815415

wu-sheng mentioned this pull request Jan 19, 2024

[Bug] Enhance VNode logic and support multiple Trace IDs in span's ref apache/skywalking#11805

Closed

3 tasks