feat(openadapt): add progress bar in `record.py` and `visualize.py` #318

KrishPatel13 · 2023-06-26T16:18:46Z

What kind of change does this PR introduce?
Feature: Add Progress Bar for Record and Visualization

Summary

Issue: #316
Checklist

My code follows the style guidelines of OpenAdapt
I have perfomed a self-review of my code
If applicable, I have added tests to prove my fix is functional/effective
I have linted my code locally prior to submission
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation (e.g. README.md, requirements.txt)
New and existing unit tests pass locally with my changes

How can your code be run and tested?

Other information

(better for USer pov)

KrishPatel13 · 2023-06-26T16:20:33Z

Quick Preview of Progress bar in Visualization: #316 (comment)

KrishPatel13 · 2023-06-26T16:20:54Z

Quick Preview of Progress Bar in Record: #316 (comment)

KrishPatel13 · 2023-06-26T16:22:10Z

Progress bar with Visualization looks good, but for record I have added the Progress bar in the function write_event

openadapt/record.py

on record.py and visualize.py

KrishPatel13 · 2023-06-26T16:29:46Z

Quick Preview of Progress Bar in Record: #316 (comment)

The messy visualization are due to threads. as the function write_event is called in threads and multiple times as and when the action enveet are added to the write_performace queue. Hence, you will notice the progress bar in multiple times during the logging of record.py.

0dm · 2023-06-26T16:57:19Z

we might need to use an event queue to ensure that there's only one progress bar. Or another approach could be to capture stdout while scrubbing (there isn't any useful info anyway it's usually just warnings about the process getting forked)

KrishPatel13 · 2023-06-26T17:10:01Z

we might need to use an event queue to ensure that there's only one progress bar

@0dm This sounds good, But on thinking: For only 1 Progress bar we need a single Loop that iterate over N times. Where N should be predetermined. If we change N then it will be difficult to think of a progress bar. Here in record.py, when writing to db, we do not have any loop which it iterated over. Hence, we might need to design something similar first.

Could you expand on how can we implement this :?

0dm · 2023-06-26T18:07:52Z

as said in meeting - we should only use the progress bar after sending termination signal - it'll be cleaner

KrishPatel13 · 2023-06-26T20:59:52Z

Agreed! Working on it

KrishPatel13 · 2023-06-27T04:32:56Z

I am finding it difficult to replicate our requirement with alive-progress bar. The reason is that when want the progress bar disabled before the termination. As soon as record is terminated we want it to be activated and then start. We can disable the progress bar in alive-progress. But currently it does not have the feature to get reactivated/enabled.

Hence, I am now tryin tqdm. I had to spent a lot of time today searching/fixing our use case. I was not able to work on extension due to this :(

KrishPatel13 · 2023-06-27T05:36:29Z

To be consistent with all of the files:

I am using TQDM Progress bar with all Record.py, Visualization.py and Scripts/scrub.py/scrub_mp4

KrishPatel13 · 2023-06-27T05:49:13Z

Final Preview for Recrod.py:

Final Preview for Visualiztion.py:

--add dynamic_cols: to enable adjustments when window is resized Order: --total -description --unit --Optional[bar_format] --colour --dynamic_ncols

KrishPatel13 · 2023-06-27T05:54:41Z

@abrichr / @0dm Ready for review!

openadapt/record.py

openadapt/visualize.py

requirements.txt

0dm · 2023-06-28T19:44:21Z

visualization is working as expected now, I will try looking into the recording issue since it's still persisting.

0dm · 2023-06-28T19:48:57Z

looks like qsize is not supported everywhere.

KrishPatel13 · 2023-06-28T20:37:54Z

Thank you @0dm.

Any alternatives you found for Mac in that case ? That serves the same purpose.

0dm · 2023-06-28T23:38:05Z

@KrishPatel13 maybe this will be of use?

https://gist.github.com/FanchenBao/d8577599c46eab1238a81857bb7277c9

-- using custom MyQueue class

KrishPatel13 · 2023-06-29T03:02:52Z

@0dm
I have added the fix for MacOs as you gave the link. Please review again 🙏

0dm · 2023-06-29T03:41:58Z

works!

openadapt/record.py

openadapt/thirdparty_customization/my_queue.py

0dm · 2023-06-29T03:50:01Z

just a few minor changes and I think we'll be good for merging

KrishPatel13

Self-reviewd!

OpenAdaptAI#318 (comment)

KrishPatel13 · 2023-06-29T04:10:02Z

@abrichr / @0dm , Ready for Merging!

KrishPatel13 · 2023-06-29T04:10:50Z

@0dm Thank your time and help ;)

* run `poetry lock --no-update` * add alive-progress via poetry and in code * add progress bar in visualization * add a check for MAX_EVENT = None * update the title for the Progress bAr (better for USer pov) * update the requirement.txt * ran ` black --line-length 80 <file>` on record.py and visualize.py * remove all progress bar from record * add tqdm progress bar in recrod.py * add tqdm for visualiztion * remove alive-progress * consistent tqdm api --add dynamic_cols: to enable adjustments when window is resized Order: --total -description --unit --Optional[bar_format] --colour --dynamic_ncols * Update requirements.txt Co-authored-by: Aaron <57018940+0dm@users.noreply.github.com> * Address comemnt: OpenAdaptAI#318 (comment) * remove incorrect indent * remove rows * try to fix distorted table in html * add custom queue class * lint --line-length 80 * fix `NotImplementedError` for MacOs -- using custom MyQueue class * rename custom -> thirdparty_customization * rename to something useful * address comments * rename dir to customized_imports * rename to extensions OpenAdaptAI#318 (comment) --------- Co-authored-by: Aaron <57018940+0dm@users.noreply.github.com>

Krish Patel added 5 commits June 26, 2023 08:57

run poetry lock --no-update

29ec8ec

add alive-progress via poetry and in code

2f29934

add progress bar in visualization

1ff5e9d

add a check for MAX_EVENT = None

790a17e

update the title for the Progress bAr

37f22c0

(better for USer pov)

KrishPatel13 requested review from abrichr and 0dm June 26, 2023 16:18

KrishPatel13 self-assigned this Jun 26, 2023

KrishPatel13 commented Jun 26, 2023

View reviewed changes

openadapt/record.py Outdated Show resolved Hide resolved

Krish Patel added 2 commits June 26, 2023 12:24

update the requirement.txt

afbaa2c

ran black --line-length 80 <file>

7227c5c

on record.py and visualize.py

Krish Patel and others added 2 commits June 26, 2023 18:14

remove all progress bar from record

0c5f4e9

Merge branch 'MLDSAI:main' into feature/record_pb

d2adac1

Krish Patel added 3 commits June 27, 2023 01:29

add tqdm progress bar in recrod.py

9e8ee4d

add tqdm for visualiztion

3de913f

remove alive-progress

e5b1601

consistent tqdm api

f7c42d1

--add dynamic_cols: to enable adjustments when window is resized Order: --total -description --unit --Optional[bar_format] --colour --dynamic_ncols

0dm requested changes Jun 28, 2023

View reviewed changes

openadapt/record.py Show resolved Hide resolved

openadapt/visualize.py Outdated Show resolved Hide resolved

requirements.txt Outdated Show resolved Hide resolved

KrishPatel13 requested a review from 0dm June 28, 2023 15:23

KrishPatel13 linked an issue Jun 28, 2023 that may be closed by this pull request

Progress bar for recording and visualization #316

Closed

Merge branch 'MLDSAI:main' into feature/record_pb

b2df437

Krish Patel added 5 commits June 28, 2023 20:02

add custom queue class

9c063ef

lint --line-length 80

857382a

fix NotImplementedError for MacOs

c6fbf3a

-- using custom MyQueue class

rename custom -> thirdparty_customization

416ce80

rename to something useful

00fdcc3

0dm requested changes Jun 29, 2023

View reviewed changes

openadapt/record.py Outdated Show resolved Hide resolved

openadapt/thirdparty_customization/my_queue.py Outdated Show resolved Hide resolved

openadapt/thirdparty_customization/my_queue.py Outdated Show resolved Hide resolved

Krish Patel added 2 commits June 28, 2023 23:56

address comments

a612535

rename dir to customized_imports

5204148

KrishPatel13 commented Jun 29, 2023

View reviewed changes

rename to extensions

e2419cc

OpenAdaptAI#318 (comment)

0dm approved these changes Jun 29, 2023

View reviewed changes

KrishPatel13 changed the title ~~Feature/record pb~~ feat(openadapt): Add Progress Bar in record.py and visualize.py Jun 29, 2023

KrishPatel13 changed the title ~~feat(openadapt): Add Progress Bar in record.py and visualize.py~~ feat(openadapt): add progress bar in record.py and visualize.py Jun 29, 2023

KrishPatel13 added 2 commits June 29, 2023 17:50

Merge branch 'OpenAdaptAI:main' into feature/record_pb

cb9a3d8

Merge branch 'main' into feature/record_pb

aa61233

abrichr merged commit 3e12fd4 into OpenAdaptAI:main Jul 3, 2023

abrichr deleted the feature/record_pb branch July 3, 2023 22:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(openadapt): add progress bar in `record.py` and `visualize.py` #318

feat(openadapt): add progress bar in `record.py` and `visualize.py` #318

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

0dm commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023

0dm commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 27, 2023

KrishPatel13 commented Jun 27, 2023

KrishPatel13 commented Jun 27, 2023 •

edited

Loading

KrishPatel13 commented Jun 27, 2023

0dm commented Jun 28, 2023

0dm commented Jun 28, 2023

KrishPatel13 commented Jun 28, 2023

0dm commented Jun 28, 2023

KrishPatel13 commented Jun 29, 2023

0dm commented Jun 29, 2023

0dm commented Jun 29, 2023

KrishPatel13 left a comment

KrishPatel13 commented Jun 29, 2023

KrishPatel13 commented Jun 29, 2023

feat(openadapt): add progress bar in record.py and visualize.py #318

feat(openadapt): add progress bar in record.py and visualize.py #318

Conversation

KrishPatel13 commented Jun 26, 2023 • edited Loading

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023 • edited Loading

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023 • edited Loading

0dm commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023

0dm commented Jun 26, 2023

KrishPatel13 commented Jun 26, 2023

KrishPatel13 commented Jun 27, 2023

KrishPatel13 commented Jun 27, 2023

KrishPatel13 commented Jun 27, 2023 • edited Loading

KrishPatel13 commented Jun 27, 2023

0dm commented Jun 28, 2023

0dm commented Jun 28, 2023

KrishPatel13 commented Jun 28, 2023

0dm commented Jun 28, 2023

KrishPatel13 commented Jun 29, 2023

0dm commented Jun 29, 2023

0dm commented Jun 29, 2023

KrishPatel13 left a comment

Choose a reason for hiding this comment

KrishPatel13 commented Jun 29, 2023

KrishPatel13 commented Jun 29, 2023

feat(openadapt): add progress bar in `record.py` and `visualize.py` #318

feat(openadapt): add progress bar in `record.py` and `visualize.py` #318

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

KrishPatel13 commented Jun 26, 2023 •

edited

Loading

KrishPatel13 commented Jun 27, 2023 •

edited

Loading