Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[close #421] add timelines tool for slow log #420

Closed
wants to merge 19 commits into from

Conversation

iosmanthus
Copy link
Member

@iosmanthus iosmanthus commented Dec 21, 2021

Signed-off-by: iosmanthus myosmanthustree@gmail.com

What problem does this PR solve?

close: #421
Increasing the read readability of the slow log, add a timeline to illustrate the slow log process.

What is changed and how it works?

Usage: <main class> [--resolution=<resolution>] [--threshold=<threshold>] <path>
      <path>   The log file to parse
      --width=<width>
               resolution of timelines
      --threshold=<threshold>
               threshold of warning time, in ms

This tool parses the existing logs of client-java and extracts the slow log, then converts them into timelines.
Three arguments are provided:

  1. path: the path of the log file.
  2. resolution: the maximum bars for a timeline, default is 70.
  3. threshold: above the threshold, the timelines print in red, default is 10ms

image

text:

function: putIfAbsent
start: 22:50:36.364
end: 22:50:37.232
region: {Region[111] ConfVer[33] Version[5] Store[13] KeyRange[key0]:[key1]}
key: key0
================================================================================  total 868ms
================================================================================  868ms callWithRetry tikvpb.Tikv/RawCompareAndSwap
=============                                                                     150ms gRPC tikvpb.Tikv/RawCompareAndSwap
                                                                                  6ms backoff BoTiKVRPC
              =============                                                       151ms gRPC tikvpb.Tikv/RawCompareAndSwap
                            =                                                     16ms backoff BoTiKVRPC
                             =============                                        150ms gRPC tikvpb.Tikv/RawCompareAndSwap
                                           ==                                     26ms backoff BoTiKVRPC
                                             =============                        150ms gRPC tikvpb.Tikv/RawCompareAndSwap
                                                           ======                 68ms backoff BoTiKVRPC
                                                                  =============   150ms gRPC tikvpb.Tikv/RawCompareAndSwap
                                                                                  0ms backoff BoTiKVRPC
                                                                                  0ms backoff BoRegionMiss

Check List for Tests

This PR has been tested by at least one of the following methods:

  • Unit test
  • Manual test (add detailed scripts or steps below)

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the difference with scripts, what kind of code is expected to be put in scripts and tools respectively?

@iosmanthus
Copy link
Member Author

what's the difference with scripts, what kind of code is expected to be put in scripts and tools respectively?

scripts might be some shell script or python script to build the project, it might be more appropriate to put the tool into tools because it generates something else and it's not a script.

@iosmanthus iosmanthus changed the title add timelines tool for slow log [clost #421] add timelines tool for slow log Dec 21, 2021
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus iosmanthus changed the title [clost #421] add timelines tool for slow log [close #421] add timelines tool for slow log Dec 21, 2021
Copy link
Collaborator

@marsishandsome marsishandsome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this PR changes the release logic?
Before: mvn clean package -DskipTests
After: build tools? release tools binary?

Copy link
Collaborator

@marsishandsome marsishandsome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend writing the tool in Java and release in the same tikv-java-client jar, so that users can run the tools directly without another executable binary.

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus iosmanthus changed the title [close #421] add timelines tool for slow log [WIP] [close #421] add timelines tool for slow log Dec 27, 2021
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus
Copy link
Member Author

@zz-jason @marsishandsome PTAL

@iosmanthus iosmanthus changed the title [WIP] [close #421] add timelines tool for slow log [close #421] add timelines tool for slow log Dec 28, 2021
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus
Copy link
Member Author

Will this PR changes the release logic? Before: mvn clean package -DskipTests After: build tools? release tools binary?

This tool will not change the release logic, it is a dependent binary tool to parse slow log instead of a library.

@iosmanthus
Copy link
Member Author

/run-all-tests

@iosmanthus
Copy link
Member Author

/run-all-tests release-5.0

@iosmanthus
Copy link
Member Author

/run-all-tests tikv = release-5.0

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
iosmanthus and others added 3 commits December 30, 2021 17:20
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus
Copy link
Member Author

@zz-jason PTAL

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus
Copy link
Member Author

@zz-jason PTAL

Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
@iosmanthus
Copy link
Member Author

@zz-jason PTAL

@iosmanthus
Copy link
Member Author

/run-all-tests

1 similar comment
@iosmanthus
Copy link
Member Author

/run-all-tests

@iosmanthus
Copy link
Member Author

@zz-jason @marsishandsome PTAL

@github-actions
Copy link

This PR is stale because it has been open 30 days with no activity. Remove the status/stale label or comment or this PR will be closed in 7 days.

@zz-jason
Copy link
Member

zz-jason commented Apr 6, 2022

I'm going to close this PR since it hasn't been updated for a long time. Feel free to reopen it once you wish to advance this PR in the future.

@zz-jason zz-jason closed this Apr 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add timelines for slow log
3 participants