diff --git a/.gitignore b/.gitignore index 2837bce..cc6934a 100644 --- a/.gitignore +++ b/.gitignore @@ -5,7 +5,7 @@ __pycache__* .DS_Store docs/.DS_Store dist/ -testgiacomovalli.egg-info/ +openhdemg.egg-info/ prove.py prove_storage.py diff --git a/CODEOWNERS b/CODEOWNERS index a1352a5..bba5f83 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -4,6 +4,6 @@ # someone opens a pull request. * @GiacomoValliPhD -# When there are changes in the GUI folder, @giacomovalliphd +# When there are changes in the GUI folder, @GiacomoValliPhD # and @PaulRitsche will be requested for review. -/openhdemg/gui/ @giacomovalliphd @PaulRitsche +/openhdemg/gui/ @GiacomoValliPhD @PaulRitsche diff --git a/README.md b/README.md index dd18d4b..a8232b0 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@

- +


@@ -23,7 +23,7 @@ Electromyography (HD-EMG) recordings. Some of its main features are listed below 7. **Save** the results of the analyses and the edited file. ## Start immediately -If you already know how to use Python, that's the way to go! Otherwise, have a look at the tutorial explaining how to [Setup your Python working environment](https://www.giacomovalli.com/openhdemg/tutorials/Setup_working_env/). +If you already know how to use Python, that's the way to go! Otherwise, have a look at the tutorial explaining how to [Setup your Python working environment](https://www.giacomovalli.com/openhdemg/tutorials/setup_working_env/). *openhdemg* can be easily installed using pip: @@ -37,7 +37,7 @@ or conda: conda install -c conda-forge openhdemg ``` -If you want an overview of what you can do with the *openhdemg* library, have a look at the [Quick Start section](https://www.giacomovalli.com/openhdemg/Quick-Start/). +If you want an overview of what you can do with the *openhdemg* library, have a look at the [Quick Start section](https://www.giacomovalli.com/openhdemg/quick-start/). ## Good to know In addition to the rich set of modules and functions presented in the **API documentation**, *openhdemg* offers also a practical graphical user interface (GUI) from which many tasks can be performed without writing a single line of code! @@ -50,7 +50,7 @@ python -m openhdemg.gui.openhdemg_gui Once opened, it will look like this. It is cool, isn't it? -![gui_preview](https://www.giacomovalli.com/openhdemg/md_graphics/Index/GUI_Preview.png) +![gui_preview](https://www.giacomovalli.com/openhdemg/md_graphics/index/gui_preview.png) ## Why openhdemg The *openhdemg* project was born in 2022 with the aim to provide the HD-EMG community with a free and open-source framework to analyse motor units' properties. diff --git a/docs/Contacts.md b/docs/Contacts.md deleted file mode 100644 index b06c0e3..0000000 --- a/docs/Contacts.md +++ /dev/null @@ -1,25 +0,0 @@ -## Primary contact - -:octicons-mail-24:   Giacomo Valli: giacomo.valli@phd.unipd.it - -## Discussion forum - -:fontawesome-brands-github:   Read answers or ask questions in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. If you are not familiar with GitHub discussions, please read this [post](https://github.com/GiacomoValliPhD/openhdemg/discussions/42){:target="_blank"}. This will allow the *openhdemg* community to answer your questions. - -## Follow us on Twitter - -:fontawesome-brands-twitter:   [https://twitter.com/openhdemg](https://twitter.com/openhdemg){:target="_blank"} - -## Meet the developers - -Giacomo Valli: - -- The creator of the project and the developer of the library. - -- Giacomo Valli obtained a master degree in Sports Science and a research fellowship in molecular biology of exercise at the University of Urbino (IT). He is currently a PhD student at the University of Padova (IT) in neuromuscular physiology. He is investigating the electrophysiological modifications happening during disuse, disease and aging and linking this information to the molecular alterations of the muscle. - -Paul Ritsche: - -- The developer of the GUI. - -- Paul Ritsche obtained a master degree in Sports Science at the University of Basel (CH). He is currently a research associate at the University of Basel (CH) focusing on muscle ultrasonography. He is investigating automatic ultrasonography image analysis methods to evaluate muscle morphological as well architectural parameters. diff --git a/docs/What's-New.md b/docs/What's-New.md deleted file mode 100644 index b0a68be..0000000 --- a/docs/What's-New.md +++ /dev/null @@ -1,5 +0,0 @@ -:octicons-tag-24: 0.1.0-beta.1   :octicons-clock-24: June 2023 - -What's new? Well, everything. This is our firt release, if you are using it, congratulations, you are a pioneer! - -Please note, this is a **beta** release, which means that a lot can change in this version and the library is not yet ready to be used without double-checking the results that you get. diff --git a/docs/about-us.md b/docs/about-us.md new file mode 100644 index 0000000..2f437c2 --- /dev/null +++ b/docs/about-us.md @@ -0,0 +1,92 @@ +## Mission + +
+ +“To build the most complete, easy-to-use, free, and open-source framework for the analysis of High-Density Electromyography (HD-EMG) recordings” + +
+ +
+But we cannot achieve this alone! That's why *openhdemg* is a community-driven project where everyone's [contribution](contribute.md) is welcomed and essential. + +## Goals + +:octicons-hash-24: community :octicons-hash-24: inclusion :octicons-hash-24: collaboration :octicons-hash-24: knowledgesharing + +We aim to cultivate a culture of knowledge sharing, collaboration, and open-source contributions within the HD-EMG community, ensuring that expertise and advancements are accessible to all. + +:octicons-hash-24: advance :octicons-hash-24: discover :octicons-hash-24: research + +We want to empower researchers with an efficient, effective and comprehensive framework for the analysis of HD-EMG recordings and single motor unit properties to advance their research. + +## Roadmap + +A roadmap is a collection of planned **milestones** and **tasks** that are necessary for the successful development and growth of the *openhdemg* project. It outlines the key steps and objectives that need to be achieved to meet the project's goals and deliver value to the community. The roadmap drives the project's evolution, ensuring that efforts are focused, organized, and aligned with the overall vision. + +At this stage, we have identified and set six major milestones for the *openhdemg* project. These milestones are divided into two categories: three on the development side and three on the engagement side. + +To read the complete roadmap, click on the next button. + +[Complete roadmap   :fontawesome-solid-map-location-dot:](about-us/complete-roadmap.md){ .md-button .md-button--primary } + +
+ +``` mermaid +graph TB; + A[Development] --> B(Feedback from beta test) + B --> C(Stable release v0.1) + C --> D(Continuous updates) + E[Engagement] --> F(Reach) + E[Engagement] --> G(Outreach) + E[Engagement] --> H(New contributors) + F --> I(Feedback) + G --> I(Feedback) + I --> D + H --> D + D --> L(Release v1.0) +``` + +## Meet the developers + +Giacomo Valli: + +- giacomo.valli@phd.unipd.it + +- The creator of the project and the developer of the library. + +- Giacomo Valli obtained a master degree in Sports Science and a research fellowship in molecular biology of exercise at the University of Urbino (IT). He is currently a PhD student at the University of Padova (IT) in neuromuscular physiology. He is investigating the electrophysiological modifications happening during disuse, disease and aging and linking this information to the molecular alterations of the muscle. + +Paul Ritsche: + +- paul.ritsche@unibas.ch + +- The developer of the GUI. + +- Paul Ritsche obtained a master degree in Sports Science at the University of Basel (CH). He is currently a research associate at the University of Basel (CH) focusing on muscle ultrasonography. He is investigating automatic ultrasonography image analysis methods to evaluate muscle morphological as well architectural parameters. + +## Meet the contributors + +Francesco Negro: + +- francesco.negro@unibs.it + +- Contribution:   :fontawesome-solid-brain: Knowledge sharing   :fontawesome-solid-file-code: Code sharing   :octicons-codescan-checkmark-24: Accuracy check + +- Francesco Negro is a Full Professor at the Department of Clinical and Experimental Sciences at Universita’ degli Studi di Brescia (IT). His research interests include applied physiology of the human motor system, signal processing of intramuscular and surface electromyography, and modeling of spinal neural networks. + +Andrea Casolo: + +- andrea.casolo@unipd.it + +- Contribution:   :fontawesome-solid-brain: Knowledge sharing   :octicons-codescan-checkmark-24: Accuracy check + +- Andrea Casolo is an Assistant Professor at the Department of Biomedical Sciences, University of Padova (IT). He obtained a MSc in Health and Physical Activity (2016) and a PhD in Human Movement and Sport Sciences (2020) from the University of Rome "Foro Italico". His research interests focus on the neural control of movement and the study of neuromuscular plasticity to physical exercise investigated with high-density surface electromyography. + +Giuseppe De Vito: + +- giuseppe.devito@unipd.it + +- Contribution:   :fontawesome-solid-brain: Knowledge sharing + +- Giuseppe De Vito is a full Professor of Human Physiology in the Department of Biomedical Sciences at University of Padova (IT). He was, from 2007 until 2019, Professor and Dean in the School of Public Health, Physiotherapy & Sports Science at University College Dublin (IE) (Head of School between 2014 and 2019). Giuseppe does research in Human and Exercise Physiology. + diff --git a/docs/about-us/complete-roadmap.md b/docs/about-us/complete-roadmap.md new file mode 100644 index 0000000..867a050 --- /dev/null +++ b/docs/about-us/complete-roadmap.md @@ -0,0 +1,109 @@ +
+ +``` mermaid +graph TB; + A[Development] --> B(Feedback from beta test) + B --> C(Stable release v0.1) + C --> D(Continuous updates) + E[Engagement] --> F(Reach) + E[Engagement] --> G(Outreach) + E[Engagement] --> H(New contributors) + F --> I(Feedback) + G --> I(Feedback) + I --> D + H --> D + D --> L(Release v1.0) +``` + +
+ +A roadmap is a collection of planned **milestones** and **tasks** that are necessary for the successful development and growth of the *openhdemg* project. It outlines the key steps and objectives that need to be achieved to meet the project's goals and deliver value to the community. The roadmap drives the project's evolution, ensuring that efforts are focused, organized, and aligned with the overall vision. + +Milestones represent significant achievements and tasks are the actionable steps to reach those milestones. They provide a structured approach to project planning and execution, tracking progress and ensuring systematic completion. The roadmap is flexible, allowing us to incorporate users feedback and refine the plan based on community input. + +## Milestones + +Legend for milestones status: + +:fontawesome-solid-check:   Completed + +:fontawesome-solid-clock:   Ongoing + +:fontawesome-solid-calendar-day:   Planned date + +At this stage, we have identified and set six major milestones for the *openhdemg* project. These milestones are divided into two categories: three on the development side and three on the engagement side. + +-------------------------------------------- + +### Development milestones + +The development milestones focus on advancing the framework's functionality, improving existing features, and introducing new algorithms and analysis techniques. These milestones aim to enhance the capabilities and performance of *openhdemg*, providing a more powerful and comprehensive tool for analyzing High-Density Electromyography (HD-EMG) recordings. + +**Going public**   :fontawesome-solid-check: + +On 04/07/2023 the *openhdemg* project was released to the public with a beta release on [PyPI](https://pypi.org/project/openhdemg/){:target="_blank"}, a public [GitHub](https://github.com/GiacomoValliPhD/openhdemg){:target="_blank"} repository, a website and a [Twitter](https://twitter.com/openhdemg){:target="_blank"} page. This milestone marks the beginning of *openhdemg* as an open-source project. + +**Stable release v0.1**   :fontawesome-solid-clock:   :fontawesome-solid-calendar-day: + +Planned by end of 2023. While a beta release is meant for testing purpose, a stable release represents the transition to a production-ready version of the framework. The primary objective of this milestone is to ensure the reliability, robustness, and usability of *openhdemg* for a wide range of users. + +**Release v1.0**   :fontawesome-solid-clock: + +The release of Version 1.0 is undoubtedly the most significant achievement for an open-source project like *openhdemg*, signifying a substantial enhancement in the project's completeness and usability. With this release, the project has achieved a level of maturity that fulfills the needs of a wide range of users in the field of HD-EMG. + +-------------------------------------------- + +### Engagement milestones + +On the engagement side, the milestones aim to enhance the reach and outreach of *openhdemg* and to bring new [contributors](#meet-the-contributors) in the project. These milestones focus on expanding the visibility and impact of *openhdemg* within the HD-EMG community and beyond. + +**Reach 1**   :fontawesome-solid-clock:   :fontawesome-solid-calendar-day: + +Planned by end of 2023. By leveraging preferred channels such as [Twitter](https://twitter.com/openhdemg){:target="_blank"} and congresses, we aim to engage with new individuals and organizations interested in HD-EMG analysis. Target to complete 'Reach 1' = 100 [Twitter](https://twitter.com/openhdemg){:target="_blank"} followers. + +**Outreach 1**   :fontawesome-solid-clock:   :fontawesome-solid-calendar-day: + +Planned by end of 2023. This milestone involves improving documentation, providing tutorials and educational resources, and enhancing user support. By achieving these milestones, we aim to empower users with the knowledge and tools they need to effectively utilize *openhdemg* and conduct their own HD-EMG analyses. Target to complete 'Outreach 1' = cover all the functionalities of the *openhdemg* framework with specific tutorials. + +**Increase contributors 1**   :fontawesome-solid-clock: + +Bringing new contributors to the *openhdemg* project is fundamental to increase the functionalities of the framework and to build a collaborative community of experts. Target to complete 'Increase contributors 1' = 5 external contributors + +## Tasks + +### Development tasks + +**Stable release v0.1** + +- Identify and address any critical bugs or issues reported during the beta testing phase. +- Conduct extensive testing on different platforms and configurations to ensure the stability and reliability of the framework. +- Incorporate user feedback and suggestions to improve the user interface, features, and overall user experience. +- Create comprehensive documentation for installation, usage, and troubleshooting of *openhdemg*. + +**Release v1.0** + +- Enhance the framework's performance and efficiency to handle larger datasets and complex analyses. +- Implement additional algorithms and analysis techniques to broaden the capabilities of *openhdemg*. +- Conduct thorough testing and validation of the framework's functionalities to ensure accuracy and reliability. +- Document and communicate the major updates and improvements in the release to the user community. + +### Engagement tasks + +**Reach 1** + +- Develop a social media strategy for *openhdemg*, including regular posting, engaging with relevant hashtags, and connecting with HD-EMG researchers and practitioners. +- Share success stories, case studies, and relevant content about *openhdemg* on Twitter to attract a wider audience and increase followers. +- Actively participate in HD-EMG-related congresses, conferences, and events to network with professionals in the field and promote *openhdemg*. + +**Outreach 1** + +- Create comprehensive tutorials and educational resources that cover various aspects of HD-EMG analysis using *openhdemg*. +- Improve the documentation to provide clear instructions, examples, and explanations of the framework's functionalities. +- Establish a user support system, such as a forum or mailing list and promote the use of the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. + +**Increase contributors 1** + +- Actively encourage contributions from the community by creating a contributor-friendly environment and providing guidance on [how to get involved](../contribute.md). +- Identify specific areas where external contributors can make meaningful contributions, such as implementing new algorithms, improving existing features, or conducting performance optimizations. +- Collaborate with potential contributors through issue discussions, pull request reviews, and effective communication channels to onboard them into the *openhdemg* community. + diff --git a/docs/API_analysis.md b/docs/api_analysis.md similarity index 100% rename from docs/API_analysis.md rename to docs/api_analysis.md diff --git a/docs/API_electrodes.md b/docs/api_electrodes.md similarity index 100% rename from docs/API_electrodes.md rename to docs/api_electrodes.md diff --git a/docs/API_info.md b/docs/api_info.md similarity index 100% rename from docs/API_info.md rename to docs/api_info.md diff --git a/docs/API_mathtools.md b/docs/api_mathtools.md similarity index 100% rename from docs/API_mathtools.md rename to docs/api_mathtools.md diff --git a/docs/API_muap.md b/docs/api_muap.md similarity index 100% rename from docs/API_muap.md rename to docs/api_muap.md diff --git a/docs/API_openfiles.md b/docs/api_openfiles.md similarity index 58% rename from docs/API_openfiles.md rename to docs/api_openfiles.md index 9611ec4..a7a6788 100644 --- a/docs/API_openfiles.md +++ b/docs/api_openfiles.md @@ -1,45 +1,32 @@ Description ----------- -This module contains all the functions that are necessary to open or save -MATLAB (.mat), JSON (.json) or custom (.csv) files.
-MATLAB files are used to store data from the DEMUSE and the OTBiolab+ -software while JSON files are used to save and load files from this -library.
-The choice of saving files in the open standard JSON file format was -preferred over the MATLAB file format since it has a better integration -with Python and has a very high cross-platform compatibility. +This module contains all the functions that are necessary to open or save MATLAB (.mat), JSON (.json) or custom (.csv) files. .mat files are currently used to store data from the DEMUSE and the OTBiolab+ software, while .csv files are used to store custom data. Instead, .json files are used to save and load files from this library.
+The choice of saving files in the open standard JSON file format was preferred over the MATLAB file format since it has a better integration with Python and has a very high cross-platform compatibility. Function's scope ---------------- - **emg_from_samplefile**:
Used to load the sample file provided with the library. - **emg_from_otb** and **emg_from_demuse**:
- Used to load .mat files coming from the DEMUSE or the OTBiolab+ - software. Demuse has a fixed file structure while the OTB file, in - order to be compatible with this library should be exported with a - strict structure as described in the function emg_from_otb. - In both cases, the input file is a .mat file. -- **refsig_from_otb**:
- Used to load files from the OTBiolab+ software that contain only - the REF_SIGNAL. + Used to load .mat files coming from the DEMUSE or the OTBiolab+ software. Demuse has a fixed file structure while the OTB file, in order to be compatible with this library should be exported with a strict structure as described in the function emg_from_otb. In both cases, the input file is a .mat file. - **emg_from_customcsv**:
Used to load custom file formats contained in .csv files. +- **refsig_from_otb** and **refsig_from_customcsv**:
+ Used to load files from the OTBiolab+ software or from a custom .csv file that contain only the REF_SIGNAL. - **save_json_emgfile**, **emg_from_json**:
- Used to save the working file to a .json file or to load the .json - file. + Used to save the working file to a .json file or to load the .json file. - **askopenfile**, **asksavefile**:
- A quick GUI implementation that allows users to select the file to - open or save. + A quick GUI implementation that allows users to select the file to open or save. Notes ----- Once opened, the file is returned as a dictionary with keys:
-"SOURCE" : source of the file (e.g., "DEMUSE", "OTB", "custom")
+"SOURCE" : source of the file (i.e., "CUSTOMCSV", "DEMUSE", "OTB")
+"FILENAME" : the name of the opened file
"RAW_SIGNAL" : the raw EMG signal
"REF_SIGNAL" : the reference signal
-"PNR" : pulse to noise ratio
-"SIL" : silouette score
+"ACCURACY" : accuracy score (depending on source file type)
"IPTS" : pulse train (decomposed source)
"MUPULSES" : instants of firing
"FSAMP" : sampling frequency
@@ -47,17 +34,22 @@ Once opened, the file is returned as a dictionary with keys:
"EMG_LENGTH" : length of the emg file (in samples)
"NUMBER_OF_MUS" : total number of MUs
"BINARY_MUS_FIRING" : binary representation of MUs firings
+"EXTRAS" : additional custom values
-The only exception is when OTB files are loaded with just the reference signal: +The only exception is when files are loaded with just the reference signal: -"SOURCE": source of the file (i.e., "OTB_refsig")
-"FSAMP": sampling frequency
-"REF_SIGNAL": the reference signal
+"SOURCE" : source of the file (i.e., "CUSTOMCSV_REFSIG", "OTB_REFSIG")
+"FILENAME" : the name of the opened file
+"FSAMP" : sampling frequency
+"REF_SIGNAL" : the reference signal
+"EXTRAS" : additional custom values
Additional informations can be found in the -[info module](API_info.md#openhdemg.library.info.info.data) and in the +[info module](api_info.md#openhdemg.library.info.info.data) and in the function's description. +Furthermore, all the users are encouraged to read the dedicated tutorial [Structure of the emgfile](tutorials/emgfile_structure.md). +
::: openhdemg.library.openfiles.emg_from_samplefile @@ -95,6 +87,13 @@ function's description.
+::: openhdemg.library.openfiles.refsig_from_customcsv + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.openfiles.save_json_emgfile options: show_root_full_path: False diff --git a/docs/API_plotemg.md b/docs/api_plotemg.md similarity index 90% rename from docs/API_plotemg.md rename to docs/api_plotemg.md index 22dadfc..dc815d4 100644 --- a/docs/API_plotemg.md +++ b/docs/api_plotemg.md @@ -1,7 +1,6 @@ Description ----------- -This module contains all the functions used to visualise the emg file, -the MUs properties or to save figures. +This module contains all the functions used to visualise the content of the imported EMG file, the MUs properties or to save figures.
diff --git a/docs/API_tools.md b/docs/api_tools.md similarity index 87% rename from docs/API_tools.md rename to docs/api_tools.md index d743908..f65b7bd 100644 --- a/docs/API_tools.md +++ b/docs/api_tools.md @@ -21,6 +21,13 @@ shortcuts necessary to operate with the HD-EMG recordings.
+::: openhdemg.library.tools.mupulses_from_binary + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.tools.resize_emgfile options: show_root_full_path: False @@ -42,6 +49,13 @@ shortcuts necessary to operate with the HD-EMG recordings.
+::: openhdemg.library.tools.delete_empty_mus + options: + show_root_full_path: False + show_root_heading: True + +
+ ::: openhdemg.library.tools.sort_mus options: show_root_full_path: False diff --git a/docs/Cite-Us.md b/docs/cite-us.md similarity index 100% rename from docs/Cite-Us.md rename to docs/cite-us.md diff --git a/docs/contacts.md b/docs/contacts.md new file mode 100644 index 0000000..7fb7c3e --- /dev/null +++ b/docs/contacts.md @@ -0,0 +1,11 @@ +## Primary contact + +:octicons-mail-24:   Any correspondence should be emailed to: openhdemg@gmail.com + +## Discussion forum + +:fontawesome-brands-github:   Read answers or ask questions in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. If you are not familiar with GitHub discussions, please read this [post](https://github.com/GiacomoValliPhD/openhdemg/discussions/42){:target="_blank"}. This will allow the *openhdemg* community to answer your questions. + +## Follow us on Twitter + +:fontawesome-brands-twitter:   [https://twitter.com/openhdemg](https://twitter.com/openhdemg){:target="_blank"} diff --git a/docs/contribute.md b/docs/contribute.md new file mode 100644 index 0000000..7094a8a --- /dev/null +++ b/docs/contribute.md @@ -0,0 +1,65 @@ +This is probably the most important page of the website. As a community, we need your contribution to grow! + +There are many ways by which you can contribute to the growth of the *openhdemg* project, and all of them are equally important. + +If you want to become part of the [team](about-us.md#meet-the-developers), read through the next sections to understand which ways of contribution fit better with your skills and expertise. For any question, or to begin the collaboration, [contact us](contacts.md) through your favorite channel. + +## Contribution categories + +:fontawesome-solid-microchip:   Code development + +This category is for individuals who want to contribute to the openhdemg project by writing code. As a developer, you can help enhance the framework's functionality, improve existing features, fix bugs, and implement new algorithms and analysis techniques. Your contributions will directly impact the usability and effectiveness of *openhdemg*. For details, read the specific section [Guidelines for code developers](#general-guidelines-for-code-developers). + + +:fontawesome-solid-brain:   Knowledge sharing + +In the knowledge sharing category, we invite individuals with specialised expertise to contribute their valuable insights and knowledge to the *openhdemg* project. By sharing your expertise, you can help others in the community gain a deeper understanding of advanced analysis techniques, coding practices, and best practices in the field of HD-EMG. Your contribution will empower others to make the most out of *openhdemg* and accelerate the progress of HD-EMG research. Knowledge sharing is crucial for the growth and advancement of the *openhdemg* community. + +:fontawesome-solid-file-code:   Code sharing + +The code sharing category is dedicated to individuals who have developed their own algorithms in Python or other languages, such as MATLAB, and want to contribute them to the *openhdemg* community. Your contribution can range from implementing new analysis techniques, data processing algorithms, or innovative visualization methods. We welcome you to share your code and contribute to the collective knowledge of the project. + +:octicons-codescan-checkmark-24:   Accuracy check + +Ensuring the accuracy and reliability of the *openhdemg* framework is paramount. In this category, you can contribute by thoroughly testing and verifying the results of the framework's analysis algorithms and functionalities and providing feedback on potential improvements or issues. In this way, you can help enhance the accuracy and validity of *openhdemg's* results and increase the credibility of the entire *openhdemg* project. + +:fontawesome-solid-bullhorn:   Promotion and advertising + +In order to grow our community, it is essential to increase our visibility and reach. Promotion and advertising play a crucial role in spreading the word about the *openhdemg* project and attracting new contributors. By actively engaging in promotion and advertising efforts, you can help us reach a wider audience and encourage more individuals to join our community. There are various ways you can contribute to promotion and advertising: + +1. Social Media: Help us promote *openhdemg* on social media platforms by sharing project updates, success stories, and relevant content. Use hashtags and tag relevant individuals or organizations to increase visibility. +2. Blogging and Content Creation: Write blog posts, articles, or tutorials about *openhdemg* and its benefits. Share your experiences, insights, and use cases to inspire others and encourage them to get involved. +3. Outreach and Collaboration: Connect with related communities, organizations, or academic institutions to collaborate on joint projects, guest blog posts, or events. By expanding our network, we can amplify our message and reach new audiences. +4. Presentations and Workshops: Offer presentations or workshops at conferences, seminars, or webinars to showcase the capabilities of *openhdemg*. Demonstrate its potential applications and engage with the audience to generate interest and curiosity. +5. Documentation and Case Studies: Contribute to the development of comprehensive documentation and case studies that highlight the value and impact of *openhdemg*. These resources will serve as references for researchers in the field. + +## Open Call for Contributions + +We welcome all enthusiastic individuals who wish to contribute to the *openhdemg* project but do not find an appropriate category for their ideas. If you have unique insights, suggestions, or contributions that can benefit the project, we encourage you to share them with us. Your diverse perspectives and ideas can drive innovation and shape the future of *openhdemg*. Together, we are a vibrant community dedicated to advancing the field. Join us and make a difference in the *openhdemg* project. + +## General guidelines for code developers + +By following these guidelines, you can ensure that your code contributions align with the project's standards and promote a smooth collaborative development process. + +1. Familiarize Yourself with the Project: Take the time to understand the *openhdemg* framework, its goals, and its existing codebase. Explore the documentation, tutorials, and code repositories to gain insights into the project's structure, coding conventions, and design principles. +2. Select an Area of Contribution: Identify an area within *openhdemg* where you can make a meaningful impact. It could be enhancing existing features, fixing bugs, implementing new algorithms, or improving the overall functionality and performance of the framework. +3. Follow Coding Best Practices: Write clean, readable, and well-documented code. Adhere to established coding standards, such as PEP 8 for Python, to maintain consistency and readability. Comment your code appropriately to enhance its understandability and maintainability. +4. Test and Validate Your Code: Thoroughly test your code to ensure its correctness and robustness. Write unit tests and integration tests to cover different scenarios and edge cases. Validate your code against real-world data and compare the results with expected outcomes. +5. Use Version Control: Utilize the version control systems Git to manage your code changes. Fork the *openhdemg* repository to create a separate copy under your GitHub account. Make your code contributions in the forked repository by creating a new branch from the current branch used for the development of new features. This allows for easy review, collaboration, and integration of your code into the *openhdemg* project. When you're ready, submit your changes as pull requests from your forked repository to the corresponding branch in the *openhdemg* repository. +6. Engage in Code Reviews: Participate in code reviews and provide constructive feedback to your fellow contributors. Actively engage in discussions, address comments and suggestions, and iteratively improve your code based on the feedback received. +7. Document Your Contributions: Document your code changes, including any new features, modifications, or improvements. Update the project's documentation or relevant documentation files to reflect your contributions accurately. +8. Communicate and Collaborate: Join the [*openhdemg* community discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"} to connect with other developers and maintain open communication channels. Collaborate with the community by sharing ideas, seeking help, or contributing to ongoing discussions. +9. Respect Licensing and Intellectual Property: Ensure that your code contributions comply with the project's chosen license. Respect intellectual property rights and avoid incorporating copyrighted code or resources without proper authorization or licensing. + +By following these guidelines, you can contribute effectively to the *openhdemg* project and help advance the field of HD-EMG analysis. Your code contributions will play a vital role in improving the framework, expanding its functionality, and enabling new research possibilities. Thank you for being a valuable member of our code development community! + +## Specific guidelines for code developers + +These guidelines should not be interpreted and must apply to all the developers. If you spot any divergence from these rules in the actual code, please suggest the author to edit the corresponding section. + +- Language: British English +- Documentation style: NumPy +- Respect PEP 8 +- Use a code checker (suggested flake8) +- Never alter the original structure of the emgfile described in the tutorial [Structure of the emgfile](/tutorials/emgfile_structure) +- Always work (i.e., Fetch/Pull request) on the current working branch. Not in the main branch. diff --git a/docs/GUI_advanced.md b/docs/gui_advanced.md similarity index 84% rename from docs/GUI_advanced.md rename to docs/gui_advanced.md index 96fb3e5..286d27d 100644 --- a/docs/GUI_advanced.md +++ b/docs/gui_advanced.md @@ -2,15 +2,16 @@ This is the toturial for the `Advanced Tools` in the *openhdemg* GUI. Great that you made it this far! In the next few sections we will take a look at the more advanced functions implemented in the GUI. But first of all, you need to click the `Advanced Tools`button in the main window of the GUI to get to the respective adavanced analysis. The `Advanced Tools Window` will open. -![advanced_analysis](md_graphics/GUI/Advanced_analysis_window.png) +![advanced_analysis](md_graphics/gui/advanced_analysis_window.png) So far, we have included three advanced analyses in the *openhdemg* GUI. - `Motor Unit Tracking` - `Duplicate Removal` -- `Conduction Velocity Calculation` +- `Conduction Velocity Estimation` For all of those, the specification of a `Matrix Orientation` and a `Matrix Code` is required. The `Matrix Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window` when using the `Plot EMG`function. The `Matrix Orientation` can be either **0** or **180** and must be chosen from the dropdown list. + The `Matrix Code` must be specified according to the one you used during acquisition. So far, the codes - `GR08MM1305` @@ -18,13 +19,13 @@ The `Matrix Code` must be specified according to the one you used during acquisi - `GR10MM0808` - `None` -are implemented. You must choose one from the respective dropdown list. -In case you selected `None`, the entrybox `Rows, Columns` will appear. Please specify the number of rows and columns of your used matrix since you now bypass included matrix codes. In example, specifying +are implemented. You must choose one from the respective dropdown list. In case you selected `None`, the entrybox `Rows, Columns` will appear. Please specify the number of rows and columns of your used matrix since you now bypass included matrix codes. `Orientation` is ignored when `Matrix Code` is `None`. In example, specifying ```Python Rows, Columns: 13, 5 ``` means that your File has 65 channels. + Once you specified these parameter, you can click the `Advaned Analysis` button to start your analysis. ----------------------------------------- @@ -32,16 +33,16 @@ Once you specified these parameter, you can click the `Advaned Analysis` button ## Motor Unit Tracking When you want to track MUs across two different files, you need to select the `Motor Unit Tracking` options and specify the `Matrix Code` and `Matrix Orentation` in the `Advanced Tools Window`. Once you clicked the `Advanced Analysis` button, the `MUs Tracking Window` will pop-up. -![mus_tracking](md_graphics/GUI/MU_Tracking_window.png) +![mus_tracking](md_graphics/gui/mu_tracking_window.png) 1. You need to specify the `Type of file` you want to track MUs across in the respective dropdown. The available filetypes are: - `OTB` (.mat file exportable by OTBiolab+) - `DEMUSE` (.mat file used in DEMUSE) - `OPENHDEMG` (emgfile or reference signal stored in .json format) - - `CUSTOM` (custom data from a .csv file) + - `CUSTOMCSV` (custom data from a .csv file) - Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the **Type of file** corresponding to the type of your file. In case you selected `OTB` specify the `extension factor` in the dropdown. + Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the **Type of file** corresponding to the type of your file. In case you selected `OTB`, specify the `extension factor` in the dropdown. 2. Load the files according to specified `Type of file`using the `Load File 1` and `Load File 2` buttons. @@ -60,26 +61,25 @@ When you want to track MUs across two different files, you need to select the `M ## Duplicate Removal When you want to remove MUs duplicates across different files, you need to select the `Duplicate Removal` options and specify the `Matrix Code` and `Matrix Orentation` in the `Advanced Tools Window`. Once you clicked the `Advanced Analysis` button, the `Duplicate Removal Window` will pop-up. `Duplicate Removal` requires similar input as `Motor Unit Tracking`, so please take a look at the [`Motor Unit Tracking`](#motor-unit-tracking) section. However, you need to do two more things. -![duplicate_removal](md_graphics/GUI/Duplicate_Removal_window.png) +![duplicate_removal](md_graphics/gui/duplicate_removal_window.png) 1. You should specify How to remove the duplicated MUs in the `Which` dropdown. You can choose between - munumber: Duplicated MUs are removed from the file with more MUs. - - PNR: The MU with the lowest PNR is removed. - - SIL: The MU with the lowest SIL is removed. + - accuracy: The MU with the lowest accuracy score is removed. 2. By clicking the `Remove Duplicates` button, you start the removal process. 3. Specify a filename and location to save the file(s) with duplicates removed in a .json format. ## Conduction Velocity -Prior to calculation of the `Conduction Velocity` you need to load a file in the main window of the GUI. Take a look at the [intro](GUI_intro.md#specifying-an-analysis-file) section. Once you have done this, open the `Advanced Tool Window` using the `Advanced Analaysis` button. +Prior to calculation of the `Conduction Velocity` you need to load a file in the main window of the GUI. Take a look at the [intro](gui_intro.md#specifying-an-analysis-file) section. Once you have done this, open the `Advanced Tool Window` using the `Advanced Analaysis` button. 1. Select `Conduction Velocity` in the `Analysis Tool` dropdown, decide on the `Matrix Orientation` and `Matrix Code` as described [above](#graphical-interface). 2. Click the `Advanced Analysis` button to start the calculation of the `Conduction Velocity`. The `MUs cv estimation` window will pop up. - ![cv_estimation](md_graphics/GUI/CV_Estimation_window.png) + ![cv_estimation](md_graphics/gui/cv_estimation_window.png) 3. In the top left of the `MUs cv estimation` window select the MU for which you want to calculate the conduction velocity using the `MU Number` dropdown. @@ -98,9 +98,10 @@ Prior to calculation of the `Conduction Velocity` you need to load a file in the -------------------------------------- -We are now at the end of describing the advanced functions included in the *openhdemg* GUI. In case you need further clarification, don't hesitate to post a question in the Github discussion forum (LINK). Moreover, if you noticed an error that was not properly catched by the GUI, please file a bug report according to our guidelines (LINK). -If you want to take a look at more basic stuff, check out the [basic](GUI_basics.md). +We are now at the end of describing the advanced functions included in the *openhdemg* GUI. If you want to take a look at more basic stuff, check out the [basic](gui_basics.md). ## More questions? We hope that this tutorial was useful. If you need any additional information, do not hesitate to read the answers or ask a question in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. If you are not familiar with GitHub discussions, please read this [post](https://github.com/GiacomoValliPhD/openhdemg/discussions/42){:target="_blank"}. This will allow the *openhdemg* community to answer your questions. + +Moreover, if you noticed an error that was not properly catched by the GUI, please [report the issue](https://github.com/GiacomoValliPhD/openhdemg/issues){:target="_blank"}. diff --git a/docs/GUI_basics.md b/docs/gui_basics.md similarity index 94% rename from docs/GUI_basics.md rename to docs/gui_basics.md index c27d9e6..ac9625f 100644 --- a/docs/GUI_basics.md +++ b/docs/gui_basics.md @@ -1,22 +1,22 @@ # Graphical Interface -This is the basic introduction to the *openhdemg* GUI. In the next few sections, we will go through the basic analysis functions embedded in the GUI. For the advanced stuff, take a look at the [advanced](GUI_advanced.md) chapter. We will start with how to sort the motor units (MUs) included in your analysis file, go over force and MU property analysis, take a detour on plotting, and take a look at how to save and reset your analysis. Have fun! +This is the basic introduction to the *openhdemg* GUI. In the next few sections, we will go through the basic analysis functions embedded in the GUI. For the advanced stuff, take a look at the [advanced](gui_advanced.md) chapter. We will start with how to sort the motor units (MUs) included in your analysis file, go over force and MU property analysis, take a detour on plotting, and take a look at how to save and reset your analysis. Have fun! -------------------------------------------- ## Motor Unit Sorting To sort the MUs included in your analysis file in order of their recruitement, we implemented a sorting algorithm. The MUs are sorted based on their recruitement order in an ascending manner. -1. Load a file. Take a look at the [intro](GUI_intro.md#specifying-an-analysis-file) section on how to do so. +1. Load a file. Take a look at the [intro](gui_intro.md#specifying-an-analysis-file) section on how to do so. -2. Pay attention to view the MUs first, using the `View MUs` button (we explained this button in the [intro](GUI_intro.md) chapter). The MUs will be sorted anyways, but without viewing them you won't see what is happening. +2. Pay attention to view the MUs first, using the `View MUs` button (we explained this button in the [intro](gui_intro.md) chapter). The MUs will be sorted anyways, but without viewing them you won't see what is happening. 3. On the left hand side in the main window of the GUI, you can find the `Sort MUs` button. It is located in row three, column two. Once you press the button, the MUs will be sorted. ## Remove Motor Units To remove MUs included in your analysis file, you can click the `Remove MUs` button. The button is located on the left hand side in the main window of the GUI in column one of row four. -![remove_mus](md_graphics/GUI/Remove_MU_window.png) +![remove_mus](md_graphics/gui/remove_mu_window.png) 1. View the MUs using the `View MUs` button prior to MU removal, you can directly see what is happening. @@ -34,7 +34,7 @@ To remove MUs included in your analysis file, you can click the `Remove MUs` but ## Reference Signal Editing The *openhdemg* GUI also allows you to edit and filter reference signals corresponding to your analysis file (this can be either a file containing both the MUs and the reference signal or a file containing only the reference signal). -![reference_sig](md_graphics/GUI/Refsig_Filter_window.png) +![reference_sig](md_graphics/gui/refsig_filter_window.png) 1. View the MUs using the `View MUs` button prior to reference signal editing, so you can see what is happening. @@ -85,7 +85,7 @@ The *openhdemg* GUI also allows you to edit and filter reference signals corresp ## Resize EMG File Sometimes, resizing of your analysis file is unevitable. Luckily, *openhdemg* provides an easy solution. In row five and column two in the left side of the GUI, you can find the `Resize File` button. -![resize_emg](md_graphics/GUI/Resize_window.png) +![resize_emg](md_graphics/gui/resize_window.png) 1. View the MUs using the `View MUs` button prior to file resizing, you can directly see what is happening. @@ -96,7 +96,7 @@ Sometimes, resizing of your analysis file is unevitable. Luckily, *openhdemg* pr ## Analyse Force Signal In order to analyse the force signal in your analysis file, you can press the `Analyse Force` button located in row six and column one in the left side of the GUI. A new pop-up window will open where you can analyse the maximum voluntary contraction (MVC) value as well as the rate of force development (RFD). -![force_analysis](md_graphics/GUI/Force_Analysis_window.png) +![force_analysis](md_graphics/gui/force_analysis_window.png) ### Maximum voluntary contraction 1. In order to get the MVC value, simply press the `Get MVC` button. A pop-up plot opens and you can select the area where you suspect the MVC to be. @@ -119,7 +119,7 @@ In order to analyse the force signal in your analysis file, you can press the `A ## Motor Unit Properties When you press the `MU Properties` button in row six and column two, the `Motor Unit Properties` Window will pop up. In this window, you have the option to analyse several MUs propierties such as the MUs recruitement threshold or the MUs discharge rate. -![mus_properties](md_graphics/GUI/MU_properties_window.png) +![mus_properties](md_graphics/gui/mu_properties_window.png) 1. Specify your priorly calculated MVC in the `Enter MVC [N]:` textbox, like @@ -181,6 +181,7 @@ Subsequently to specifying the MVC, you can calculate a number of basic MUs prop - The absolute/relative recruitment/derecruitment thresholds - The discharge rate at recruitment, derecruitment, during the steady-state phase and during the entire contraction +- The individual and average accuracy - The coefficient of variation of interspike interval - The coefficient of variation of force signal @@ -201,7 +202,7 @@ and are all displayed in the `Result Output` once the analysis in completed. In *openhdemg* we have implemented options to plot your analysis file ... a lot of options! Upon clicking the `Plot MUs` button, the `Plot Window` will pop up. In the top right corner of the window, you can find an information button forwarding you directly to some tutorials. -![plot_mus](md_graphics/GUI/Plot_window.png) +![plot_mus](md_graphics/gui/plot_window.png) You can choose between the follwing plotting options: @@ -237,7 +238,7 @@ These three setting options are universally used in all plots. There are two mor ``` means that your File has 65 channels. -2. You need to specify the `Orientation` in row two and column four in the left side of the `Plot Window`. The `Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window`. +2. You need to specify the `Orientation` in row two and column four in the left side of the `Plot Window`. The `Orientaion` must match the one of your matrix during acquisition. You can find a reference image for the `Orientation` at the bottom in the right side of the `Plot Window`. `Orientation` is ignored when `Matrix Code` is `None`. ### Plot Raw EMG Signal 1. Click the `Plot EMGsig` button in row four and column one in the left side of the `Plot Window`, to plot the raw emg signal of your analysis file. @@ -298,9 +299,10 @@ We all make mistakes! But, most likely, we are also able to correct them. In cas -------------------------------------------- -We hope you had fun! We are now at the end of describing the basic functions included in the *openhdemg* GUI. In case you need further clarification, don't hesitate to post a question in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. Moreover, if you noticed an error that was not properly catched by the GUI, please file a bug report according to our guidelines (LINK). -If you want to proceed to the advanced stuff now, take a look at the [advanced](GUI_advanced.md) tab on the left side of the webpage. +If you want to proceed to the advanced stuff now, take a look at the [advanced](gui_advanced.md) tab on the left side of the webpage. ## More questions? We hope that this tutorial was useful. If you need any additional information, do not hesitate to read the answers or ask a question in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. If you are not familiar with GitHub discussions, please read this [post](https://github.com/GiacomoValliPhD/openhdemg/discussions/42){:target="_blank"}. This will allow the *openhdemg* community to answer your questions. + +Moreover, if you noticed an error that was not properly catched by the GUI, please [report the issue](https://github.com/GiacomoValliPhD/openhdemg/issues){:target="_blank"}. diff --git a/docs/GUI_intro.md b/docs/gui_intro.md similarity index 85% rename from docs/GUI_intro.md rename to docs/gui_intro.md index 018405e..0542f12 100644 --- a/docs/GUI_intro.md +++ b/docs/gui_intro.md @@ -1,13 +1,20 @@ # Graphical Interface Welcome, to the *openhdemg* Graphical User Interface (GUI) introduction! -The *openhdemg* GUI incorporates all relevant high-level functions of the *openhdemg* library. The GUI allows you to successfully perform High-Density Electromyography (HD-EMG) data anlysis **without any programming skills required**. Moreover, there is no downside to using the GUI even if you are an experienced programmer. + +The *openhdemg* GUI incorporates all relevant high-level functions of the *openhdemg* library. The GUI allows you to successfully perform High-Density Electromyography (HD-EMG) data anlysis **without any programming skills required**. Moreover, there is no downside to using the GUI even if you are an experienced programmer. + +The GUI can be simply accessed from the command line with: + +```shell +python -m openhdemg.gui.openhdemg_gui +``` ------------------------------------------------- Let us shortly walk you through the main window of the GUI. An image of the starting page of the GUI is displayed below. -![gui_preview](md_graphics/Index/GUI_Preview.png) +![gui_preview](md_graphics/index/gui_preview.png) This is your starting point for every analysis. On the left hand side you can find all the entryboxes and buttons relevant for the analyses you want to perform. In the middle you can see the plotting canvas where plots of the HD-EMG data analysis are displayed. On the right hand side you can find information buttons leading you directly to more information, tutorials, and more. And, with a little swoosh of magic, the results window appears at the bottom of the GUI once an analysis is finished. @@ -21,9 +28,10 @@ This is your starting point for every analysis. On the left hand side you can fi - `DEMUSE` (.mat file used in DEMUSE) - `OTB_REFSIG` (Reference signal in the .mat file exportable by OTBiolab+) - `OPENHDEMG` (emgfile or reference signal stored in .json format) - - `CUSTOM` (custom data from a .csv file) + - `CUSTOMCSV` (custom data from a .csv file) + - `CUSTOMCSV_REFSIG` (Reference signal in a custom .csv file) - Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the `Type of file` corresponding to the type of your file. + Each filetype corresponds to a distinct datatype that should match the file you want to analyse. So, select the `Type of file` corresponding to the type of your file. In case you selected `OTB`, specify the `extension factor` in the dropdown. 2. To actually load the file, click the **Load File** button and select the file you want to analyse. In case of occurence, follow the error messages and repeat this and the previos step. @@ -32,11 +40,12 @@ This is your starting point for every analysis. On the left hand side you can fi ## Viewing an analysis file It doesn't get any simpler than this! + Once a file is successfully loaded as described above, you can click the `View MUs` button to plot/view your file. In the middle section of the GUI, a plot containing your data should appear. ---------------------------------------- -In the two sections above, we described the two most rudimental functions in the GUI. To learn more about basic and more advanced analysis features of the GUI, check out the [basic](GUI_basics.md) and [advanced](GUI_advanced.md) chapters. +In the two sections above, we described the two most rudimental functions in the GUI. To learn more about basic and more advanced analysis features of the GUI, check out the [basic](gui_basics.md) and [advanced](gui_advanced.md) chapters. ## More questions? diff --git a/docs/index.md b/docs/index.md index 8d5aeb0..2f6171e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,7 +1,7 @@ # Welcome to openhdemg
-![banner logo](md_graphics/Index/Banner_Logo.png) +![banner logo](md_graphics/index/banner_logo.png)
@@ -20,7 +20,7 @@ 7. **Save** the results of the analyses and the edited file. ## Start immediately -If you already know how to use Python, that's the way to go! Otherwise, have a look at the tutorial explaining how to [Setup your Python environment](tutorials/Setup_working_env.md). +If you already know how to use Python, that's the way to go! Otherwise, have a look at the tutorial explaining how to [Setup your Python environment](tutorials/setup_working_env.md). *openhdemg* can be easily installed using pip: @@ -34,7 +34,7 @@ or conda: conda install -c conda-forge openhdemg ``` -If you want an overview of what you can do with the *openhdemg* library, have a look at the [Quick Start](Quick-Start.md) section and then explore all the functions in the **API reference**. +If you want an overview of what you can do with the *openhdemg* library, have a look at the [Quick Start](quick-start.md) section and then explore all the functions in the **API reference**. ## Good to know In addition to the rich set of modules and functions presented in the **API reference**, *openhdemg* offers also a practical graphical user interface (GUI) from which many tasks can be performed without writing a single line of code! @@ -47,7 +47,7 @@ python -m openhdemg.gui.openhdemg_gui Once opened, it will look like this. It is cool, isn't it? -![gui_preview](md_graphics/Index/GUI_Preview.png) +![gui_preview](md_graphics/index/gui_preview.png) ## Why openhdemg The *openhdemg* project was born in 2022 with the aim to provide the HD-EMG community with a free and open-source framework to analyse motor units' properties. diff --git a/docs/md_graphics/GUI/Advanced_analysis_window.png b/docs/md_graphics/GUI/Advanced_analysis_window.png deleted file mode 100644 index ed0dd98..0000000 Binary files a/docs/md_graphics/GUI/Advanced_analysis_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/CV_Estimation_window.png b/docs/md_graphics/GUI/CV_Estimation_window.png deleted file mode 100644 index 0b46236..0000000 Binary files a/docs/md_graphics/GUI/CV_Estimation_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Duplicate_Removal_window.png b/docs/md_graphics/GUI/Duplicate_Removal_window.png deleted file mode 100644 index 4a1d13a..0000000 Binary files a/docs/md_graphics/GUI/Duplicate_Removal_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Force_Analysis_window.png b/docs/md_graphics/GUI/Force_Analysis_window.png deleted file mode 100644 index a2592b1..0000000 Binary files a/docs/md_graphics/GUI/Force_Analysis_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/MU_Tracking_window.png b/docs/md_graphics/GUI/MU_Tracking_window.png deleted file mode 100644 index 01c7785..0000000 Binary files a/docs/md_graphics/GUI/MU_Tracking_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/MU_properties_window.png b/docs/md_graphics/GUI/MU_properties_window.png deleted file mode 100644 index fca8c80..0000000 Binary files a/docs/md_graphics/GUI/MU_properties_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Plot_window.png b/docs/md_graphics/GUI/Plot_window.png deleted file mode 100644 index 54f3a87..0000000 Binary files a/docs/md_graphics/GUI/Plot_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Refsig_Filter_window.png b/docs/md_graphics/GUI/Refsig_Filter_window.png deleted file mode 100644 index 370fdef..0000000 Binary files a/docs/md_graphics/GUI/Refsig_Filter_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Remove_MU_window.png b/docs/md_graphics/GUI/Remove_MU_window.png deleted file mode 100644 index 1563149..0000000 Binary files a/docs/md_graphics/GUI/Remove_MU_window.png and /dev/null differ diff --git a/docs/md_graphics/GUI/Resize_window.png b/docs/md_graphics/GUI/Resize_window.png deleted file mode 100644 index 4b48c11..0000000 Binary files a/docs/md_graphics/GUI/Resize_window.png and /dev/null differ diff --git a/docs/md_graphics/Index/Banner_Logo.png b/docs/md_graphics/Index/Banner_Logo.png deleted file mode 100644 index 0f9de04..0000000 Binary files a/docs/md_graphics/Index/Banner_Logo.png and /dev/null differ diff --git a/docs/md_graphics/Index/GUI_Preview.png b/docs/md_graphics/Index/GUI_Preview.png deleted file mode 100644 index 55c7826..0000000 Binary files a/docs/md_graphics/Index/GUI_Preview.png and /dev/null differ diff --git a/docs/md_graphics/gui/advanced_analysis_window.png b/docs/md_graphics/gui/advanced_analysis_window.png new file mode 100644 index 0000000..f8db494 Binary files /dev/null and b/docs/md_graphics/gui/advanced_analysis_window.png differ diff --git a/docs/md_graphics/gui/cv_estimation_window.png b/docs/md_graphics/gui/cv_estimation_window.png new file mode 100644 index 0000000..74a1714 Binary files /dev/null and b/docs/md_graphics/gui/cv_estimation_window.png differ diff --git a/docs/md_graphics/gui/duplicate_removal_window.png b/docs/md_graphics/gui/duplicate_removal_window.png new file mode 100644 index 0000000..dc2307e Binary files /dev/null and b/docs/md_graphics/gui/duplicate_removal_window.png differ diff --git a/docs/md_graphics/gui/force_analysis_window.png b/docs/md_graphics/gui/force_analysis_window.png new file mode 100644 index 0000000..af434d2 Binary files /dev/null and b/docs/md_graphics/gui/force_analysis_window.png differ diff --git a/docs/md_graphics/gui/mu_properties_window.png b/docs/md_graphics/gui/mu_properties_window.png new file mode 100644 index 0000000..7dee165 Binary files /dev/null and b/docs/md_graphics/gui/mu_properties_window.png differ diff --git a/docs/md_graphics/gui/mu_tracking_window.png b/docs/md_graphics/gui/mu_tracking_window.png new file mode 100644 index 0000000..02eaca6 Binary files /dev/null and b/docs/md_graphics/gui/mu_tracking_window.png differ diff --git a/docs/md_graphics/gui/plot_window.png b/docs/md_graphics/gui/plot_window.png new file mode 100644 index 0000000..8753830 Binary files /dev/null and b/docs/md_graphics/gui/plot_window.png differ diff --git a/docs/md_graphics/gui/refsig_filter_window.png b/docs/md_graphics/gui/refsig_filter_window.png new file mode 100644 index 0000000..2fc33e0 Binary files /dev/null and b/docs/md_graphics/gui/refsig_filter_window.png differ diff --git a/docs/md_graphics/gui/remove_mu_window.png b/docs/md_graphics/gui/remove_mu_window.png new file mode 100644 index 0000000..fe79a04 Binary files /dev/null and b/docs/md_graphics/gui/remove_mu_window.png differ diff --git a/docs/md_graphics/gui/resize_window.png b/docs/md_graphics/gui/resize_window.png new file mode 100644 index 0000000..753c22c Binary files /dev/null and b/docs/md_graphics/gui/resize_window.png differ diff --git a/docs/md_graphics/index/banner_logo.png b/docs/md_graphics/index/banner_logo.png new file mode 100644 index 0000000..3940698 Binary files /dev/null and b/docs/md_graphics/index/banner_logo.png differ diff --git a/docs/md_graphics/index/gui_preview.png b/docs/md_graphics/index/gui_preview.png new file mode 100644 index 0000000..4dc0c77 Binary files /dev/null and b/docs/md_graphics/index/gui_preview.png differ diff --git a/docs/md_graphics/Quick-Start/IDR_mu2removed.png b/docs/md_graphics/quick-start/idr_mu2removed.png similarity index 100% rename from docs/md_graphics/Quick-Start/IDR_mu2removed.png rename to docs/md_graphics/quick-start/idr_mu2removed.png diff --git a/docs/md_graphics/Quick-Start/MUs_pulses_non_sorted.png b/docs/md_graphics/quick-start/mus_pulses_non_sorted.png similarity index 100% rename from docs/md_graphics/Quick-Start/MUs_pulses_non_sorted.png rename to docs/md_graphics/quick-start/mus_pulses_non_sorted.png diff --git a/docs/md_graphics/Quick-Start/MUs_pulses_sorted.png b/docs/md_graphics/quick-start/mus_pulses_sorted.png similarity index 100% rename from docs/md_graphics/Quick-Start/MUs_pulses_sorted.png rename to docs/md_graphics/quick-start/mus_pulses_sorted.png diff --git a/docs/md_graphics/Quick-Start/MUs_pulses_sorted_idr.png b/docs/md_graphics/quick-start/mus_pulses_sorted_idr.png similarity index 100% rename from docs/md_graphics/Quick-Start/MUs_pulses_sorted_idr.png rename to docs/md_graphics/quick-start/mus_pulses_sorted_idr.png diff --git a/docs/md_graphics/Quick-Start/MUs_pulses_sorted_idr_filteroffs.png b/docs/md_graphics/quick-start/mus_pulses_sorted_idr_filteroffs.png similarity index 100% rename from docs/md_graphics/Quick-Start/MUs_pulses_sorted_idr_filteroffs.png rename to docs/md_graphics/quick-start/mus_pulses_sorted_idr_filteroffs.png diff --git a/docs/Quick-Start.md b/docs/quick-start.md similarity index 80% rename from docs/Quick-Start.md rename to docs/quick-start.md index d773004..539fc0d 100644 --- a/docs/Quick-Start.md +++ b/docs/quick-start.md @@ -40,7 +40,7 @@ Great, we are now ready to exploit all the functionalities of the library! In this example, we will use the sample file provided with *openhdemg*. -This can be simply loaded calling the function [emg_from_samplefile](API_openfiles.md#openhdemg.library.openfiles.emg_from_samplefile). +This can be simply loaded calling the function [emg_from_samplefile](api_openfiles.md#openhdemg.library.openfiles.emg_from_samplefile). ```Python # Import the library with the short name 'emg' @@ -52,7 +52,7 @@ emgfile = emg.emg_from_samplefile() *emgfile* is organised as a Python dictionary and contains different elements (which are labelled by keys). -For a full list of keys contained in the *emgfile* refer to the [openfiles documentation](API_openfiles.md#notes). +For a full list of keys contained in the *emgfile* refer to the [openfiles documentation](api_openfiles.md#notes). Each element in the emgfile can be accessed as `emgfile["element"]`. @@ -101,7 +101,7 @@ This is an extremely important information if you want to manipulate the content Apart from accessing the numerical values, we can also plot them. -In this case we are interested in visualising the MUs firing times together with the reference signal. This can be done with the function [plot_mupulses](API_plotemg.md#openhdemg.library.plotemg.plot_mupulses). +In this case we are interested in visualising the MUs firing times together with the reference signal. This can be done with the function [plot_mupulses](api_plotemg.md#openhdemg.library.plotemg.plot_mupulses). ```Python # Import the library with the short name 'emg' @@ -114,11 +114,11 @@ emgfile = emg.emg_from_samplefile() emg.plot_mupulses(emgfile=emgfile) ``` -![MUs_pulses_non_sorted](md_graphics/Quick-Start/MUs_pulses_non_sorted.png) +![MUs_pulses_non_sorted](md_graphics/quick-start/mus_pulses_non_sorted.png) Looks good, but I would rather have the MUs ordered by recruitment order and also with thinner lines! -We can do that with the function [sort_mus](API_tools.md#openhdemg.library.tools.sort_mus) and changing the parameters in [plot_mupulses](API_plotemg.md#openhdemg.library.plotemg.plot_mupulses). +We can do that with the function [sort_mus](api_tools.md#openhdemg.library.tools.sort_mus) and changing the parameters in [plot_mupulses](api_plotemg.md#openhdemg.library.plotemg.plot_mupulses). ```Python # Import the library with the short name 'emg' @@ -134,11 +134,11 @@ emgfile = emg.sort_mus(emgfile=emgfile) emg.plot_mupulses(emgfile=emgfile, linewidths=0.4) ``` -![MUs_pulses_sorted](md_graphics/Quick-Start/MUs_pulses_sorted.png) +![MUs_pulses_sorted](md_graphics/quick-start/mus_pulses_sorted.png)
-Are you curious about the dicharge rate of the MUs? You can view that with the function [plot_idr](API_plotemg.md#openhdemg.library.plotemg.plot_idr). +Are you curious about the dicharge rate of the MUs? You can view that with the function [plot_idr](api_plotemg.md#openhdemg.library.plotemg.plot_idr). ```Python # Import the library with the short name 'emg' @@ -154,15 +154,15 @@ emgfile = emg.sort_mus(emgfile=emgfile) emg.plot_idr(emgfile=emgfile) ``` -![MUs_pulses_sorted_idr](md_graphics/Quick-Start/MUs_pulses_sorted_idr.png) +![MUs_pulses_sorted_idr](md_graphics/quick-start/mus_pulses_sorted_idr.png) ## 4. Edit the reference signal The MUs look quite good; however, the reference signal is a bit noisy and the offset is not to 0. -The noise can be removed filtering the reference signal with the function [filter_refsig](API_tools.md#openhdemg.library.tools.filter_refsig) that, by default, applies a 4th order, zero-lag, low-pass Butterworth filter with a cutoff frequency of 15 Hz. +The noise can be removed filtering the reference signal with the function [filter_refsig](api_tools.md#openhdemg.library.tools.filter_refsig) that, by default, applies a 4th order, zero-lag, low-pass Butterworth filter with a cutoff frequency of 15 Hz. -Instead, the offset can be removed with the function [remove_offset](API_tools.md#openhdemg.library.tools.remove_offset) that automatically detects the offset based on a number of samples at the beginning of the recording. +Instead, the offset can be removed with the function [remove_offset](api_tools.md#openhdemg.library.tools.remove_offset) that automatically detects the offset based on a number of samples at the beginning of the recording. ```Python # Import the library with the short name 'emg' @@ -185,7 +185,7 @@ emgfile = emg.remove_offset(emgfile=emgfile, auto=1024) emg.plot_idr(emgfile=emgfile) ``` -![MUs_pulses_sorted_idr_filteroffs](md_graphics/Quick-Start/MUs_pulses_sorted_idr_filteroffs.png) +![MUs_pulses_sorted_idr_filteroffs](md_graphics/quick-start/mus_pulses_sorted_idr_filteroffs.png) ## 5. Remove unwanted MUs @@ -193,9 +193,9 @@ There might be cases in which we need to remove one or more MUs from our *emgfil From the visual inspection of our plots, we can see that the firings pattern of MU number 2 (remember, Python is in base 0!!!) is not really regular. We might therefore have doubts about its quality. -A way to assess the quality of the MUs is to look at the separation between the signal and the noise. This is efficiently measured by the silouette (SIL) score. +A way to assess the quality of the MUs is to look at the separation between the signal and the noise. This is efficiently measured by accuracy scores. -This score is automatically calculated while importing the *emgfile* and can be easily accessed as `emgfile["SIL"]`. +This score is automatically calculated while importing the *emgfile* and can be easily accessed as `emgfile["ACCURACY"]`. In our sample file, the accuracy is calculated by the Silhouette (SIL) score (Negro 2016). ```Python # Import the library with the short name 'emg' @@ -205,7 +205,7 @@ import openhdemg.library as emg emgfile = emg.emg_from_samplefile() # Print the SIL score -print(emgfile["SIL"]) +print(emgfile["ACCURACY"]) """Output 0 @@ -217,9 +217,9 @@ print(emgfile["SIL"]) """ ``` -Our suspicion was right, MU number 2 has the lowest SIL score. +Our suspicion was right, MU number 2 has the lowest accuracy score. -In order to remove this MU, we can use the function [delete_mus](API_tools.md#openhdemg.library.tools.delete_mus). +In order to remove this MU, we can use the function [delete_mus](api_tools.md#openhdemg.library.tools.delete_mus). ```Python # Import the library with the short name 'emg' @@ -245,15 +245,15 @@ emgfile = emg.delete_mus(emgfile=emgfile, munumber=2) emg.plot_idr(emgfile=emgfile) ``` -![IDR_mu2removed](md_graphics/Quick-Start/IDR_mu2removed.png) +![IDR_mu2removed](md_graphics/quick-start/idr_mu2removed.png) ## 6. Analyse fundamental MUs properties Now that we removed the unwanted MUs and adjusted the reference signal, we can proceed with the analysis of some fundamental MUs properties like the thresholds of recruitment and derecruitment and the discharge rate. -In the past, this used to require many lines of code, but thanks to *openhdemg*, we can now do that with 1 line of code using the function [basic_mus_properties](API_analysis.md#openhdemg.library.analysis.basic_mus_properties). +In the past, this used to require many lines of code, but thanks to *openhdemg*, we can now do that with 1 line of code using the function [basic_mus_properties](api_analysis.md#openhdemg.library.analysis.basic_mus_properties). -After calling the function [basic_mus_properties](API_analysis.md#openhdemg.library.analysis.basic_mus_properties), the user will be asked to select the start and the end of the steady-state phase. This can be done positioning the mouse on the desired point and then pressing a keybord key (such as 'a'). To remove points, right click with your mouse. +After calling the function [basic_mus_properties](api_analysis.md#openhdemg.library.analysis.basic_mus_properties), the user will be asked to select the start and the end of the steady-state phase. This can be done positioning the mouse on the desired point and then pressing a keybord key (such as 'a'). To remove points, right click with your mouse. ```Python # Import the library with the short name 'emg' @@ -285,35 +285,29 @@ results = emg.basic_mus_properties( print(results) """ - MVC MU_number PNR avg_PNR SIL avg_SIL abs_RT \ -0 634.0 0 27.480307 29.877575 0.899082 0.922923 30.621759 -1 NaN 1 28.946493 NaN 0.919601 NaN 32.427026 -2 NaN 2 28.640680 NaN 0.917190 NaN 68.371911 -3 NaN 3 34.442821 NaN 0.955819 NaN 118.504004 - - abs_DERT rel_RT rel_DERT DR_rec DR_derec DR_start_steady \ -0 36.168135 4.829930 5.704753 7.548770 5.449581 11.788779 -1 31.167703 5.114673 4.916041 8.344515 5.333535 11.254445 -2 67.308703 10.784213 10.616515 5.699017 3.691367 9.007505 -3 102.761472 18.691483 16.208434 5.701081 4.662196 7.393645 - - DR_end_steady DR_all_steady DR_all COVisi_steady COVisi_all \ -0 10.401857 11.154952 10.693076 6.833642 19.104306 -1 9.999033 10.751960 10.543011 8.364553 15.408739 -2 7.053079 8.168471 7.949294 10.097045 23.324503 -3 6.430807 6.908502 6.814687 11.211862 16.319474 - - COV_steady -0 1.422424 -1 NaN -2 NaN -3 NaN + MVC MU_number ACCURACY avg_ACCURACY abs_RT abs_DERT \ +0 634.0 0 0.899082 0.922923 30.621759 36.168135 +1 NaN 1 0.919601 NaN 32.427026 31.167703 +2 NaN 2 0.917190 NaN 68.371911 67.308703 +3 NaN 3 0.955819 NaN 118.504004 102.761472 + + rel_RT rel_DERT DR_rec DR_derec DR_start_steady DR_end_steady \ +0 4.829930 5.704753 7.548770 5.449581 11.788779 10.401857 +1 5.114673 4.916041 8.344515 5.333535 11.254445 9.999033 +2 10.784213 10.616515 5.699017 3.691367 9.007505 7.053079 +3 18.691483 16.208434 5.701081 4.662196 7.393645 6.430807 + + DR_all_steady DR_all COVisi_steady COVisi_all COV_steady +0 11.154952 10.693076 6.833642 19.104306 1.422424 +1 10.751960 10.543011 8.364553 15.408739 NaN +2 8.168471 7.949294 10.097045 23.324503 NaN +3 6.908502 6.814687 11.211862 16.319474 NaN """ ``` ## 7. Save the results and the edited file -It looks like we got a lot of results, which makes it extremely inefficient to copy them manually. +It looks like we got a lot of results, which makes of it extremely inefficient to copy them manually. Obviously, this can be automated using one attribute of the *results* object and we can conveniently save all the results in a .csv file. @@ -349,7 +343,7 @@ results = emg.basic_mus_properties( results.to_csv("C:/Users/.../Desktop/Results.csv") ``` -Our results are now safe but, additionally, we might want to save also the *emgfile* with all the changes that we made. This can be easily done with the function [asksavefile](API_openfiles.md#openhdemg.library.openfiles.asksavefile) that will save your *emgfile* in the open standard JSON file format which has a better integration with Python and has a very high cross-platform compatibility. +Our results are now safe but, additionally, we might want to save also the *emgfile* with all the changes that we made. This can be easily done with the function [asksavefile](api_openfiles.md#openhdemg.library.openfiles.asksavefile) that will save your *emgfile* in the open standard JSON file format which has a better integration with Python and has a very high cross-platform compatibility. ```Python # Import the library with the short name 'emg' diff --git a/docs/tutorials/Setup_working_env/add_folder_toworks.png b/docs/tutorials/Setup_working_env/add_folder_toworks.png deleted file mode 100644 index 6870656..0000000 Binary files a/docs/tutorials/Setup_working_env/add_folder_toworks.png and /dev/null differ diff --git a/docs/tutorials/Setup_working_env/create_newfile.png b/docs/tutorials/Setup_working_env/create_newfile.png deleted file mode 100644 index 46dff79..0000000 Binary files a/docs/tutorials/Setup_working_env/create_newfile.png and /dev/null differ diff --git a/docs/tutorials/Setup_working_env/first_script.png b/docs/tutorials/Setup_working_env/first_script.png deleted file mode 100644 index d5c5082..0000000 Binary files a/docs/tutorials/Setup_working_env/first_script.png and /dev/null differ diff --git a/docs/tutorials/Setup_working_env/python_extension.png b/docs/tutorials/Setup_working_env/python_extension.png deleted file mode 100644 index 9735362..0000000 Binary files a/docs/tutorials/Setup_working_env/python_extension.png and /dev/null differ diff --git a/docs/tutorials/emgfile_structure.md b/docs/tutorials/emgfile_structure.md new file mode 100644 index 0000000..c1389c8 --- /dev/null +++ b/docs/tutorials/emgfile_structure.md @@ -0,0 +1,280 @@ +## What is the emgfile + +The `emgfile` is the basic data structure of the *openhdemg* framework. In practical terms, it is a Python object containing all the information of the decomposed HD-EMG file loaded in the [working environment](setup_working_env.md) via the dedicated *openhdemg* functions. + +For example: + +```Python +# Import the library with the short name 'emg' +import openhdemg.library as emg + +# Load the sample file and assign it to the emgfile object +emgfile = emg.emg_from_samplefile() +``` + +Loads the decomposed sample file provided with *openhdemg* and assigns its content to the object `emgfile`. + +## Structure of the emgfile + +The `emgfile` has a simple structure. Indeed, it is a Python dictionary with keys: + +```Python +# Import the library with the short name 'emg' +import openhdemg.library as emg + +# Load the sample file and assign it to the emgfile object +emgfile = emg.emg_from_samplefile() + +# Visualise the type of the emgfile +print(type(emgfile)) + +"""Output + +""" + +# Visualise the keys of the emgfile dictionary +print(emgfile.keys()) + +"""Output +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) +""" +``` + +That means that the `emgfile` contains the following keys (or variables, in simpler terms): + +- "SOURCE" : source of the file (i.e., "CUSTOMCSV", "DEMUSE", "OTB") +- "RAW_SIGNAL" : the raw EMG signal +- "REF_SIGNAL" : the reference signal +- "ACCURACY" : accuracy score (depending on source file type) +- "IPTS" : pulse train (decomposed source) +- "MUPULSES" : instants of firing +- "FSAMP" : sampling frequency +- "IED" : interelectrode distance +- "EMG_LENGTH" : length of the emg file (in samples) +- "NUMBER_OF_MUS" : total number of MUs +- "BINARY_MUS_FIRING" : binary representation of MUs firings +- "EXTRAS" : additional custom values + +Each key has a specific content and structure that will be presented in the next code block. + +It must be noted that some of these keys might be empty (e.g., absence of a reference signal) and, therefore, the specific content of each key must be assessed from case to case. This can be simply done taking advantage of the info class: + +```Python +# Import the library with the short name 'emg' +import openhdemg.library as emg + +# Load the sample file and assign it to the emgfile object +emgfile = emg.emg_from_samplefile() + +# Obtain info on the content of the emgfile +info = emg.info() +info.data(emgfile) + +"""Output +Data structure of the emgfile +----------------------------- + +emgfile type is: + + +emgfile keys are: +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING', 'EXTRAS']) + +Any key can be acced as emgfile[key]. + +emgfile['SOURCE'] is a of value: +OTB + +emgfile['FILENAME'] is a of value: +otb_testfile.mat + +MUST NOTE: emgfile from OTB has 64 channels, from DEMUSE 65 (includes empty channel). +emgfile['RAW_SIGNAL'] is a of value: + 0 1 2 3 4 5 6 7 8 ... 55 56 57 58 59 60 61 62 63 +0 10.172526 5.086263 12.715657 11.189778 9.155273 8.138021 9.155273 13.224284 2.034505 ... 7.120768 6.612142 10.172526 8.138021 10.681152 2.034505 14.750163 4.577637 11.698405 +1 14.750163 8.138021 12.715657 12.715657 10.681152 6.612142 13.732910 16.276041 3.051758 ... 4.577637 3.560384 11.698405 7.120768 10.681152 0.508626 10.681152 4.069010 11.698405 +2 6.103516 1.017253 6.103516 15.767415 6.103516 3.051758 6.103516 11.698405 2.034505 ... 1.525879 1.525879 3.560384 -1.017253 4.069010 -4.577637 8.138021 -1.525879 5.086263 +3 -3.051758 -7.120768 -3.051758 4.577637 -4.069010 -8.138021 -2.543132 2.543132 -7.120768 ... -8.646647 -9.155273 -3.560384 -9.155273 -6.103516 -13.732910 -1.017253 -11.698405 -2.543132 +4 -11.189778 -15.767415 -15.767415 -5.086263 -11.698405 -13.732910 -7.120768 -3.560384 -12.207031 ... -15.767415 -18.310547 -12.207031 -12.715657 -11.189778 -17.293295 -8.646647 -17.293295 -11.189778 +... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... +66555 11.189778 17.801920 16.276041 17.801920 2.034505 22.379557 8.646647 14.750163 14.750163 ... -2.034505 0.508626 2.034505 2.034505 13.224284 0.000000 10.172526 10.172526 17.801920 +66556 12.715657 22.888184 21.362305 20.853678 10.172526 26.448568 12.207031 19.836426 16.276041 ... 2.034505 7.120768 4.577637 8.646647 14.241536 5.086263 18.819174 16.276041 16.276041 +66557 6.103516 7.120768 12.207031 12.715657 0.508626 16.276041 3.051758 9.663899 5.594889 ... -5.594889 -1.525879 -6.103516 -1.525879 6.103516 -1.525879 7.629395 8.646647 8.646647 +66558 -9.663899 -9.663899 -7.120768 -7.629395 -14.241536 -1.017253 -14.750163 -7.629395 -10.681152 ... -23.905436 -17.801920 -22.888184 -20.853678 -10.681152 -17.801920 -13.224284 -8.646647 -9.155273 +66559 0.508626 1.017253 0.000000 4.577637 -2.543132 6.612142 -3.051758 1.525879 -2.034505 ... -12.715657 -6.612142 -14.750163 -10.172526 0.000000 -6.103516 1.017253 -3.051758 -2.543132 + +[66560 rows x 64 columns] + +emgfile['REF_SIGNAL'] is a of value: + 0 +0 1.640534 +1 1.660370 +2 1.700043 +3 1.719879 +4 1.739716 +... ... +66555 1.462006 +66556 1.481842 +66557 1.501679 +66558 1.481842 +66559 1.481842 + +[66560 rows x 1 columns] + +emgfile['ACCURACY'] is a of value: + 0 +0 0.879079 +1 0.955819 +2 0.917190 +3 0.899082 +4 0.919601 + +emgfile['IPTS'] is a of value: + 0 1 2 3 4 +0 -1.208628e-04 3.242122e-09 0.000004 0.000186 0.000038 +1 -4.316778e-04 -8.801236e-05 -0.000053 0.000095 -0.000063 +2 -6.112677e-06 -1.158515e-04 0.000619 -0.000379 0.000035 +3 5.843681e-05 -1.308242e-05 -0.000033 -0.000001 -0.000168 +4 4.365258e-06 5.634868e-05 -0.000423 -0.000112 0.000001 +... ... ... ... ... ... +66555 -6.366898e-08 0.000000e+00 0.000000 0.000000 0.000000 +66556 -5.846209e-05 0.000000e+00 0.000000 0.000000 0.000000 +66557 -4.853265e-05 0.000000e+00 0.000000 0.000000 0.000000 +66558 1.100347e-05 0.000000e+00 0.000000 0.000000 0.000000 +66559 1.217465e-04 0.000000e+00 0.000000 0.000000 0.000000 + +[66560 rows x 5 columns] + +emgfile['MUPULSES'] is a of length depending on total MUs number. +MUPULSES for each MU can be accessed as emgfile['MUPULSES'][MUnumber]. + +emgfile['MUPULSES'][0] is a of value: +[ 4990 6659 8310 8581 9424 9662 9905 10046 10200 10543 10762 11179 + 11469 11743 11973 12243 12580 12795 13038 13258 13396 13642 13890 14161 + 14357 14672 14975 15250 15564 15920 16281 16572 16837 17182 17321 17634 + 17973 18999 19388 19906 20178 20331 20523 20797 21020 21321 21705 21863 + 22190 22283 22656 23297 23397 23445 23598 24045 24206 24430 24678 24764 + 24982 25095 25277 25939 26581 26942 27652 28303 28515 28602 28893 29136 + 29363 30399 30604 31277 32005 32227 32780 33030 33255 33550 34239 34889 + 35261 37394 37932 38439 39061 39564 40393 41056 41919 43357 43742 44039 + 44335 44721 45182 45913 46083 46745 47076 47343 47635 47976 48147 48578 + 48880 49428 49742 49887 50516 50850 50957 51194 51350 51678 52110 52892 + 53161 53390 53795 54154 54386 54823 55032 55283 55653 56026 56282 56538 + 56931 57578 57871 58429 59077] + +emgfile['FSAMP'] is a of value: +2048.0 + +emgfile['IED'] is a of value: +8.0 + +emgfile['EMG_LENGTH'] is a of value: +66560 + +emgfile['NUMBER_OF_MUS'] is a of value: +5 + +emgfile['BINARY_MUS_FIRING'] is a of value: + 0 1 2 3 4 +0 0.0 0.0 0.0 0.0 0.0 +1 0.0 0.0 0.0 0.0 0.0 +2 0.0 0.0 0.0 0.0 0.0 +3 0.0 0.0 0.0 0.0 0.0 +4 0.0 0.0 0.0 0.0 0.0 +... ... ... ... ... ... +66555 0.0 0.0 0.0 0.0 0.0 +66556 0.0 0.0 0.0 0.0 0.0 +66557 0.0 0.0 0.0 0.0 0.0 +66558 0.0 0.0 0.0 0.0 0.0 +66559 0.0 0.0 0.0 0.0 0.0 + +[66560 rows x 5 columns] + +emgfile['EXTRAS'] is a of value: +Empty DataFrame +Columns: [0] +Index: [] +""" +``` + +As you can see, `info.data(emgfile)` provides you with all the information regarding the content of the `emgfile`, on how it is structured and on how each element can be accessed. This information is crucial to interact with the `emgfile` and, understanding its structure, is fundamental to performed advanced customisation of the functions and to exploit the full flexibility/customisability of the *openhdemg* framework. + +Please take some time to learn this! + +## Alternative structures of the emgfile + +At the moment, the only alternative to the basic `emgfile` structure is reserved for the loading of those files containing only the reference signal. This can be, for example, the case of the force signal used to calculate the maximum voluntary contraction (MVC) value. + +In this case, the `emg_refsig` is a Python dictionary with the following keys: + +- "SOURCE": source of the file (i.e., "CUSTOMCSV_REFSIG", "OTB_REFSIG") +- "FSAMP": sampling frequency +- "REF_SIGNAL": the reference signal +- "EXTRAS" : additional custom values + +## Modify the emgfile to fit your needs + +This is a fundamental part of this tutorial. You can modify the `emgfile` as you wish, but there are 2 simple rules that must be followed to allow a seamless integration with the *openhdemg* functions. + +1. Do not alter the `emgfile` keys. You should not add or remove keys and you should not alter the data type under each key. If you need to do so, please remember that some of the built-in functions might not work anymore and you might encounter unexpected errors. +2. Preserve data structures. If there is missing data you should fill the `emgfile` keys with the original data structure. The original data structure is the one presented in section [Structure of the emgfile](#structure-of-the-emgfile). For example, the reference signal is by default contained in a pd.DataFrame. Therefore, if the reference signal is absent, the dict key "REF_SIGNAL" should contain an empty pd.DataFrame. + +To modify the `emgfile` you can simply act as for modifying any Python dictionary: + +```Python +# Import the necessary libraries +import openhdemg.library as emg + +import pandas as pd +import numpy as np + +# Load the sample file and assign it to the emgfile object +emgfile = emg.emg_from_samplefile() + +# Visualise the keys of the emgfile dictionary +print(emgfile.keys()) + +"""Output +dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) +""" + +# Visualise the original data structure contained in the 'REF_SIGNAL' key +print(type(emgfile['REF_SIGNAL'])) + +"""Output + +""" + +# Replace the current 'REF_SIGNAL' with a random reference signal. +random_data = np.random.randint(0 ,20, size=(emgfile['EMG_LENGTH'], 1)) +rand_ref = pd.DataFrame(random_data, columns=[0]) +emgfile['REF_SIGNAL'] = rand_ref + +print(emgfile['REF_SIGNAL']) + +"""Output + 0 +0 2 +1 15 +2 13 +3 18 +4 3 +... .. +66555 13 +66556 11 +66557 13 +66558 1 +66559 7 +""" +``` + +## Create your own emgfile + +*openhdemg* offers a number of built-in functions to load the data from different sources. However, there might be special circumnstances that require more flexibility. In this case, the user can create custom functions to load any type of decomposed HD-EMG files. However, in order to interact with the *openhdemg* functions, the `emgfile` loaded with any custom function must respect the original [structure](#structure-of-the-emgfile). + +In case your decomposed HD-EMG file does not contain a specific variable, you should mantain the original `emgfile` keys and the original data structure, although empty. + + +## More questions? + +We hope that this tutorial was useful. If you need any additional information, do not hesitate to read the answers or ask a question in the [*openhdemg* discussion section](https://github.com/GiacomoValliPhD/openhdemg/discussions){:target="_blank"}. If you are not familiar with GitHub discussions, please read this [post](https://github.com/GiacomoValliPhD/openhdemg/discussions/42){:target="_blank"}. This will allow the *openhdemg* community to answer your questions. \ No newline at end of file diff --git a/docs/tutorials/Setup_working_env.md b/docs/tutorials/setup_working_env.md similarity index 93% rename from docs/tutorials/Setup_working_env.md rename to docs/tutorials/setup_working_env.md index 68e2ece..2541353 100644 --- a/docs/tutorials/Setup_working_env.md +++ b/docs/tutorials/setup_working_env.md @@ -2,7 +2,7 @@ Welcome to the tutorial on setting up your working environment. -This is the first step necessary to start using *openhdemg*, both if you want to use the library's functions and develop your scripts, or if you only want to use the [graphical user interface (GUI)](../GUI_intro.md). +This is the first step necessary to start using *openhdemg*, both if you want to use the library's functions and develop your scripts, or if you only want to use the [graphical user interface (GUI)](../gui_intro.md). The working environment refers to the set of resources necessary to carry out a particular task or job. In the context of this tutorial, we refer to the combination of a computer, a programming language, an integrated development environment and a set of algorithms. @@ -18,7 +18,7 @@ The integrated development environment is a software that facilitates to write, By following the steps outlined in this tutorial, you will be able to install Python, set up VS Code, and configure your environment to write clean and efficient Python code. -![vscode_overview](Setup_working_env/vscode_overview.png) +![vscode_overview](setup_working_env/vscode_overview.png) ## Install Python and VS Code on Windows @@ -67,9 +67,9 @@ Now that VS Code is installed, you need to set it up to code in Python and to in 1. Open Visual Studio Code. 2. Install the "Python" extension by Microsoft. To do so, click on the Extensions view on the left sidebar, search for "Python" in the search bar, and click the "Install" button next to the "Python" extension by Microsoft. -![python_extension](Setup_working_env/python_extension.png) +![python_extension](setup_working_env/python_extension.png) -Once the extension is installed, you need to create a folder where you will palce all the scripts that you will write. You can create this folder with any name an in any location in your computer, but make it simple to find! In this tutorial the folder was placed in the Desktop and named `Test_folder`. +Once the extension is installed, you need to create a folder where you will place all the scripts that you will write. You can create this folder with any name an in any location in your computer, but make it simple to find! In this tutorial the folder was placed in the Desktop and named `Test_folder`. Once the folder is created: @@ -77,7 +77,7 @@ Once the folder is created: 2. In VS Code click on 'Terminal' and then on 'New Terminal'. 3. Select your folder in the window that pops up. -![add_folder_toworks](Setup_working_env/add_folder_toworks.png) +![add_folder_toworks](setup_working_env/add_folder_toworks.png) With your powershell terminal that is **pointing to your folder path**, you can now create a Virtual environment. @@ -97,7 +97,7 @@ For Mac users: python3 -m venv myvenv ``` -![venv_command](Setup_working_env/venv_command.png) +![venv_command](setup_working_env/venv_command.png) This command will create a Virtual environment named `myvenv`. @@ -113,7 +113,7 @@ For Mac users: source myvenv/bin/activate ``` -![activated_venv](Setup_working_env/activated_venv.png) +![activated_venv](setup_working_env/activated_venv.png) If everything was successful, you should see the colourful name of your Virtual environment to the left of your folder path (as in the figure above). @@ -137,19 +137,19 @@ python -m openhdemg.gui.openhdemg_gui And the GUI will start: -![openhdemg GUI](../md_graphics/Index/GUI_Preview.png) +![openhdemg GUI](../md_graphics/index/gui_preview.png) If you instead want to write your own script using the functions contained in *openhdemg*, follow these steps: 1. click on the Explorer view on the left sidebar. 2. Click on the icon to create a new file in your workspace folder (`Test_folder` in this case). -3. Name the file as you wish but wit a .py extension. +3. Name the file as you wish but with a .py extension. -![create_newfile](Setup_working_env/create_newfile.png) +![create_newfile](setup_working_env/create_newfile.png) Now you can open your .py file, write your code and execute it. -![first_script](Setup_working_env/first_script.png) +![first_script](setup_working_env/first_script.png) ## Troubleshooting diff --git a/docs/tutorials/Setup_working_env/activated_venv.png b/docs/tutorials/setup_working_env/activated_venv.png similarity index 100% rename from docs/tutorials/Setup_working_env/activated_venv.png rename to docs/tutorials/setup_working_env/activated_venv.png diff --git a/docs/tutorials/setup_working_env/add_folder_toworks.png b/docs/tutorials/setup_working_env/add_folder_toworks.png new file mode 100644 index 0000000..155fb51 Binary files /dev/null and b/docs/tutorials/setup_working_env/add_folder_toworks.png differ diff --git a/docs/tutorials/setup_working_env/create_newfile.png b/docs/tutorials/setup_working_env/create_newfile.png new file mode 100644 index 0000000..b1dda39 Binary files /dev/null and b/docs/tutorials/setup_working_env/create_newfile.png differ diff --git a/docs/tutorials/setup_working_env/first_script.png b/docs/tutorials/setup_working_env/first_script.png new file mode 100644 index 0000000..a6d67be Binary files /dev/null and b/docs/tutorials/setup_working_env/first_script.png differ diff --git a/docs/tutorials/setup_working_env/python_extension.png b/docs/tutorials/setup_working_env/python_extension.png new file mode 100644 index 0000000..03a130d Binary files /dev/null and b/docs/tutorials/setup_working_env/python_extension.png differ diff --git a/docs/tutorials/Setup_working_env/venv_command.png b/docs/tutorials/setup_working_env/venv_command.png similarity index 100% rename from docs/tutorials/Setup_working_env/venv_command.png rename to docs/tutorials/setup_working_env/venv_command.png diff --git a/docs/tutorials/Setup_working_env/vscode_overview.png b/docs/tutorials/setup_working_env/vscode_overview.png similarity index 100% rename from docs/tutorials/Setup_working_env/vscode_overview.png rename to docs/tutorials/setup_working_env/vscode_overview.png diff --git a/docs/what's-new.md b/docs/what's-new.md new file mode 100644 index 0000000..4e321d9 --- /dev/null +++ b/docs/what's-new.md @@ -0,0 +1,48 @@ +## :octicons-tag-24: 0.1.0-beta.2 +:octicons-clock-24: September 2023 + +This release introduces important changes. It is mainly addressing the necessity of maximum flexibility and easy integration with any custom or proprietary file source. This release is not backward compatible. + +MAJOR CHANGES: + +- **Accuracy Measurement:** Replaced the double accuracy measures in the `emgfile` (i.e., “SIL” and “PNR”) with a single accuracy measure named “ACCURACY.” For files containing the decomposed source (also named “IPTS”), the “ACCURACY” variable will contain the silhouette score (Negro et al. 2016). For files that do not contain the decomposed source, the accuracy will be the original (often proprietary) accuracy estimate. This allows for maximum flexibility and is fundamental to interface the *openhdemg* library with any proprietary and custom implementation of the different decomposition algorithms currently available. + + To accommodate this change, all the functions in the `openfile` module have been updated. Consequently, the functions using the “SIL” or “PNR” variables have also been modified. Specifically: + + - The `basic_mus_properties` function has a new input parameter (i.e., “accuracy”) to customize the returned accuracy estimate. + - In the function `remove_duplicates_between`, the input parameter “which” now only accepts “munumber” and “accuracy” instead of “munumber,” “SIL,” and “PNR.” + +- **EXTRAS Variable:** Introduced a new “EXTRAS” variable to store any custom information in the opened file. This will be accessible in the `emgfile` dictionary with the “EXTRAS” key. This variable must contain a pd.DataFrame structure and will be preserved when saving the file. This change extends the customisability of the `emgfile`. + +- **Handling Missing Variables:** Replaced “np.nan” with empty "pd.DataFrame” for missing variables upon import of files. This change ensures consistency and avoids compatibility issues with other functions. + +- **File Import Restriction:** Restricted flexibility in the import of files. To import decomposed HD-EMG files, these must contain at least the raw EMG signal and one of the times of discharge of each MU ("MUPULSES") or their binary representation. This change ensures consistency and avoids compatibility issues with other functions. + +**OTHER CHANGES:** + +- **Sampling Frequency** and **Interelectrode Distance:** Sampling frequency and interelectrode distance are now represented by float point values to accommodate different source files. + +- **emg_from_customcsv** and **emg_from_otb:** Improved robustness and flexibility, with the possibility to load custom information in “EXTRAS.” + +- **emg_from_demuse:** Improved robustness and flexibility. + +- **New Functions:** + - `refsig_from_customcsv` to load the reference signal from a custom .csv file. + - `delete_empty_mus` to delete all the MUs without firings. + +- **Exposed Function:** Exposed `mupulses_from_binary` to extract the times of firing from the binary representation of MUs firings. + +- **Dependency Management:** Addressed reported functioning issues related to external dependencies invoked by *openhdemg*. Stricter rules have been adopted in the setup.py file for automatically installing the correct version of these dependencies. + +- **Bug Fixes:** + - Fixed a BUG in the GUI when saving results in Excel files. The bug was due to changes in newer pandas versions. + - Fixed a BUG in the function “sort_mus” when empty MUs were present. + +
+ +## :octicons-tag-24: 0.1.0-beta.1 +:octicons-clock-24: June 2023 + +What's new? Well, everything. This is our first release, if you are using it, congratulations, you are a pioneer! + +Please note, this is a **beta** release, which means that a lot can change in this version and the library is not yet ready to be used without double-checking the results that you get. diff --git a/docs/yml_graphics/Transp_Icon.png b/docs/yml_graphics/transp_icon.png similarity index 100% rename from docs/yml_graphics/Transp_Icon.png rename to docs/yml_graphics/transp_icon.png diff --git a/mkdocs.yml b/mkdocs.yml index c74ed95..ce6d35f 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -4,14 +4,31 @@ site_author: Giacomo Valli & Paul Ritsche site_description: >- All you need to know about openhdemg +# Repository +repo_name: openhdemg +repo_url: https://github.com/GiacomoValliPhD/openhdemg + copyright: Copyright © 2022 - 2023 Giacomo Valli & Paul Ritsche theme: name: "material" - logo: yml_graphics/Transp_Icon.png - favicon: yml_graphics/Transp_Icon.png + logo: yml_graphics/transp_icon.png + favicon: yml_graphics/transp_icon.png features: - content.code.copy + palette: + # Palette toggle for light mode + - media: "(prefers-color-scheme: light)" + scheme: default + toggle: + icon: material/brightness-7 + name: Switch to dark mode + # Palette toggle for dark mode + - media: "(prefers-color-scheme: dark)" + scheme: slate + toggle: + icon: material/brightness-4 + name: Switch to light mode # Select a plugin to get the docs from docstrings and setup the numpy stile as # the default style. @@ -37,27 +54,31 @@ plugins: # front of the filename whose title you want to change. nav: - Welcome: index.md - - Quick-Start.md + - About us: about-us.md + - Quick-Start: quick-start.md - Tutorials: #- Overview on the tutorials: Tutorials.md - For new users: - - Setup working environment: tutorials/Setup_working_env.md - - Graphical-Interface: - - intro: GUI_intro.md - - basics: GUI_basics.md - - advanced: GUI_advanced.md - - What's-New.md - - Contacts.md - - Cite-Us.md - - 'API Reference': - - openfiles: API_openfiles.md - - plotemg: API_plotemg.md - - analysis: API_analysis.md - - tools: API_tools.md - - mathtools: API_mathtools.md - - muap: API_muap.md - - electrodes: API_electrodes.md - - info: API_info.md + - Setup working environment: tutorials/setup_working_env.md + - Basics of openhdemg: + - Structure of the emgfile: tutorials/emgfile_structure.md + - Graphical Interface: + - Intro: gui_intro.md + - Basics: gui_basics.md + - Advanced: gui_advanced.md + - What's New: what's-new.md + - Contacts: contacts.md + - Cite us: cite-us.md + - Contribute: contribute.md + - API Reference: + - openfiles: api_openfiles.md + - plotemg: api_plotemg.md + - analysis: api_analysis.md + - tools: api_tools.md + - mathtools: api_mathtools.md + - muap: api_muap.md + - electrodes: api_electrodes.md + - info: api_info.md # Extensions markdown_extensions: @@ -70,7 +91,7 @@ markdown_extensions: - toc: permalink: true title: On this page - toc_depth: 2 + toc_depth: 3 - pymdownx.arithmatex: generic: true - pymdownx.betterem: @@ -108,9 +129,9 @@ extra: generator: false # Remove 'Made with Material for MkDocs' in the footer. social: - icon: fontawesome/brands/github - link: https://github.com/GiacomoValliPhD + link: https://github.com/GiacomoValliPhD/openhdemg - icon: fontawesome/brands/twitter - link: https:// + link: https://twitter.com/openhdemg version: provider: mike # TODO versioning diff --git a/openhdemg/__init__.py b/openhdemg/__init__.py index a71b333..8a5e814 100644 --- a/openhdemg/__init__.py +++ b/openhdemg/__init__.py @@ -1,3 +1,3 @@ __all__ = ["__version__"] -__version__ = "0.1.0-beta.1" +__version__ = "0.1.0-beta.2" diff --git a/openhdemg/gui/openhdemg_gui.py b/openhdemg/gui/openhdemg_gui.py index 8da01f4..5652783 100644 --- a/openhdemg/gui/openhdemg_gui.py +++ b/openhdemg/gui/openhdemg_gui.py @@ -78,7 +78,7 @@ class emgGUI: String containing the path to EMG file selected for analysis. self.filetype : str String containing the filetype of import EMG file. - Filetype can be "OENHDEMG", "OTB", "DEMUSE", or "OTB_REFSIG", "CUSTOM". + Filetype can be "OPENHDEMG", "OTB", "DEMUSE", "OTB_REFSIG", "CUSTOMCSV", "CUSTOMCSV_REFSIG". self.filter_order : int, default 4 The filter order. self.firings_rec : int, default 4 @@ -295,7 +295,7 @@ class emgGUI: Method do display extension factor combobx when filetype loaded is OTB. open_emgfile1() - Method to open EMG file based on the selected file type and extension factor. + Method to open EMG file based on the selected file type and extension factor. open_emgfile2() Method to open EMG file based on the selected file type and extension factor. track_mus() @@ -360,7 +360,7 @@ def __init__(self, master): # Specify Signal self.filetype = StringVar() - signal_value = ("OPENHDEMG", "OTB", "DEMUSE", "OTB_REFSIG", "CUSTOM") + signal_value = ("OPENHDEMG", "OTB", "DEMUSE", "OTB_REFSIG", "CUSTOMCSV", "CUSTOMCSV_REFSIG") signal_entry = ttk.Combobox( self.left, text="Signal", width=10, textvariable=self.filetype ) @@ -515,7 +515,7 @@ def __init__(self, master): fg_color="LightBlue4", command=lambda: ( ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/GUI_intro/") + webbrowser.open("https://www.giacomovalli.com/openhdemg/gui_intro/") ), ), ) @@ -535,7 +535,7 @@ def __init__(self, master): fg_color="LightBlue4", command=lambda: ( ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/tutorials/Setup_working_env/") + webbrowser.open("https://www.giacomovalli.com/openhdemg/tutorials/setup_working_env/") ), ), ) @@ -555,7 +555,7 @@ def __init__(self, master): fg_color="LightBlue4", command=lambda: ( ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/Contacts/#meet-the-developers") + webbrowser.open("https://www.giacomovalli.com/openhdemg/about-us/#meet-the-developers") ), ), ) @@ -575,7 +575,7 @@ def __init__(self, master): fg_color="LightBlue4", command=lambda: ( ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/Contacts/") + webbrowser.open("https://www.giacomovalli.com/openhdemg/contacts/") ), ), ) @@ -596,7 +596,7 @@ def __init__(self, master): command=lambda: ( # Check user OS for pdf opening ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/Cite-Us/") + webbrowser.open("https://www.giacomovalli.com/openhdemg/cite-us/") ), ), ) @@ -618,10 +618,10 @@ def get_file_input(self): emg_from_demuse, emg_from_otb, refsig_from_otb and emg_from_json in library. """ try: - if self.filetype.get() in ["OTB", "DEMUSE", "OPENHDEMG", "CUSTOM"]: + if self.filetype.get() in ["OTB", "DEMUSE", "OPENHDEMG", "CUSTOMCSV"]: # Check filetype for processing if self.filetype.get() == "OTB": - # Ask user to select the file + # Ask user to select the decomposed file file_path = filedialog.askopenfilename( title="Open OTB file", filetypes=[("MATLAB files", "*.mat")] ) @@ -655,12 +655,12 @@ def get_file_input(self): else: # Ask user to select the file file_path = filedialog.askopenfilename( - title="Open CUSTOM file", - filetypes=[("CSV files", ".csv")], + title="Open CUSTOMCSV file", + filetypes=[("CSV files", "*.csv")], ) self.file_path = file_path - # load refsig + # load file self.resdict = openhdemg.emg_from_customcsv(filepath=self.file_path) # Get filename @@ -679,27 +679,41 @@ def get_file_input(self): ) ttk.Label(self.left, text=str(self.resdict["EMG_LENGTH"])).grid( column=2, row=4, sticky=(W, E) - ) + ) + """ + # BUG with "OPENHDEMG" type we identify all files saved from openhdemg, + regardless of the content. This will result in an error for ttk.Label + self.resdict["NUMBER_OF_MUS"] and self.resdict["EMG_LENGTH"]. + """ else: - # Ask user to select the file - file_path = filedialog.askopenfilename( - title="Open REFSIG file", - filetypes=[ - ("MATLAB files", "*.mat"), - ("JSON files", "*.json"), - ], - ) - self.file_path = file_path + # Ask user to select the refsig file + if self.filetype.get() == "OTB_REFSIG": + file_path = filedialog.askopenfilename( + title="Open OTB_REFSIG file", + filetypes=[("MATLAB files", "*.mat")], + ) + self.file_path = file_path + # load refsig + self.resdict = openhdemg.refsig_from_otb(filepath=self.file_path) + + else: # CUSTOMCSV_REFSIG + file_path = filedialog.askopenfilename( + title="Open CUSTOMCSV_REFSIG file", + filetypes=[("CSV files", "*.csv")], + ) + self.file_path = file_path + # load refsig + self.resdict = openhdemg.refsig_from_customcsv(filepath=self.file_path) + # Get filename filename = os.path.splitext(os.path.basename(file_path))[0] self.filename = filename # Add filename to label self.master.title(self.filename) - # load refsig - self.resdict = openhdemg.refsig_from_otb(filepath=self.file_path) - # Recondifgure labels for refsig + + # Reconfigure labels for refsig ttk.Label( self.left, text=str(len(self.resdict["REF_SIGNAL"].columns)) ).grid(column=2, row=2, sticky=(W, E)) @@ -821,7 +835,7 @@ def export_to_excel(self): if hasattr(self, "mu_thresholds"): self.mu_thresholds.to_excel(writer, sheet_name="MU Thresholds") - writer.save() + writer.close() except IndexError: tk.messagebox.showerror( @@ -860,7 +874,7 @@ def reset_analysis(self): # user decided to rest analysis try: # reload original file - if self.filetype.get() in ["OTB", "DEMUSE", "OPENHDEMG", "CUSTOM"]: + if self.filetype.get() in ["OTB", "DEMUSE", "OPENHDEMG", "CUSTOMCSV"]: if self.filetype.get() == "OTB": self.resdict = openhdemg.emg_from_otb( filepath=self.file_path, @@ -875,7 +889,7 @@ def reset_analysis(self): elif self.filetype.get() == "OPENHDEMG": self.resdict = openhdemg.emg_from_json(filepath=self.file_path) - elif self.filetype.get() == "CUSTOM": + elif self.filetype.get() == "CUSTOMCSV": self.resdict = openhdemg.emg_from_customcsv( filepath=self.file_path ) @@ -893,7 +907,11 @@ def reset_analysis(self): else: # load refsig - self.resdict = openhdemg.refsig_from_otb(filepath=self.file_path) + if self.filetype.get() == "OTB_REFSIG": + self.resdict = openhdemg.refsig_from_otb(filepath=self.file_path) + else: # CUSTOMCSV_REFSIG + self.resdict = openhdemg.refsig_from_customcsv(filepath=self.file_path) + # Recondifgure labels for refsig ttk.Label( self.left, text=str(len(self.resdict["REF_SIGNAL"].columns)) @@ -1049,7 +1067,7 @@ def in_gui_plotting(self, plot="idr"): plot_refsig, plot_idr in the library. """ try: - if self.filetype.get() == "OTB_REFSIG": + if self.filetype.get() in ["OTB_REFSIG", "CUSTOMCSV_REFSIG"]: self.fig = openhdemg.plot_refsig( emgfile=self.resdict, showimmediately=False, tight_layout=True ) @@ -1792,7 +1810,10 @@ def compute_mu_dr(self): tk.messagebox.showerror("Information", "Load file prior to computation.") except ValueError: - tk.messagebox.showerror("Information", "Enter valid Firings value.") + tk.messagebox.showerror( + "Information", + "Enter valid Firings value or select a correct number of points." + ) except AssertionError: tk.messagebox.showerror("Information", "Specify Event and/or Type.") @@ -1835,7 +1856,10 @@ def basic_mus_properties(self): tk.messagebox.showerror("Information", "Load file prior to computation.") except ValueError: - tk.messagebox.showerror("Information", "Enter valid MVC.") + tk.messagebox.showerror( + "Information", + "Enter valid MVC or select a correct number of points." + ) except AssertionError: tk.messagebox.showerror("Information", "Specify Event and/or Type.") @@ -2073,7 +2097,7 @@ def plot_emg(self): fg_color="LightBlue4", command=lambda: ( ( - webbrowser.open("https://www.giacomovalli.com/openhdemg/GUI_basics/#plot-motor-units") + webbrowser.open("https://www.giacomovalli.com/openhdemg/gui_basics/#plot-motor-units") ), ), ) @@ -2531,7 +2555,7 @@ def advanced_analysis(self): head_title = "Duplicate Removal Window" else: head_title = "Conduction Velocity Window" - + self.head = tk.Toplevel(bg="LightBlue4") self.head.title(head_title) self.head.iconbitmap( @@ -2541,7 +2565,7 @@ def advanced_analysis(self): # Specify Signal self.filetype_adv = StringVar() - signal_value = ("OTB", "DEMUSE", "OPENHDEMG", "CUSTOM") + signal_value = ("OTB", "DEMUSE", "OPENHDEMG", "CUSTOMCSV") signal_entry = ttk.Combobox( self.head, text="Signal", width=8, textvariable=self.filetype_adv ) @@ -2646,7 +2670,7 @@ def advanced_analysis(self): self.which_adv = StringVar() which_combobox = ttk.Combobox( self.head, - values=["munumber", "SIL", "PNR"], + values=["munumber", "accuracy"], textvariable=self.which_adv, state="readonly", width=8, diff --git a/openhdemg/library/__init__.py b/openhdemg/library/__init__.py index 7c0658f..8352b61 100644 --- a/openhdemg/library/__init__.py +++ b/openhdemg/library/__init__.py @@ -14,6 +14,7 @@ emg_from_demuse, refsig_from_otb, emg_from_customcsv, + refsig_from_customcsv, save_json_emgfile, emg_from_json, askopenfile, diff --git a/openhdemg/library/analysis.py b/openhdemg/library/analysis.py index 7604c1f..86ca7f2 100644 --- a/openhdemg/library/analysis.py +++ b/openhdemg/library/analysis.py @@ -15,7 +15,7 @@ def compute_thresholds(emgfile, event_="rt_dert", type_="abs_rel", mvc=0): """ Calculates recruitment/derecruitment thresholds. - Values are calculated both in absolute and relative therms. + Values are calculated both in absolute and relative terms. Parameters ---------- @@ -56,7 +56,7 @@ def compute_thresholds(emgfile, event_="rt_dert", type_="abs_rel", mvc=0): contraction. - compute_covisi : calculate the coefficient of variation of interspike interval. - - compute_drvariability : claculate the DR variability. + - compute_drvariability : calculate the DR variability. Examples -------- @@ -230,7 +230,7 @@ def compute_dr( contraction. - compute_covisi : calculate the coefficient of variation of interspike interval. - - compute_drvariability : claculate the DR variability. + - compute_drvariability : calculate the DR variability. Notes ----- @@ -431,6 +431,7 @@ def basic_mus_properties( n_firings_steady=10, start_steady=-1, end_steady=-1, + accuracy="default", mvc=0, ): """ @@ -444,6 +445,8 @@ def basic_mus_properties( the coefficient of variation of interspike interval, the coefficient of variation of force signal. + Accuracy measures, MVC and steadiness are also returned. + Parameters ---------- emgfile : dict @@ -458,6 +461,19 @@ def basic_mus_properties( The start and end point (in samples) of the steady-state phase. If < 0 (default), the user will need to manually select the start and end of the steady-state phase. + accuracy : str {"default", "SIL", "PNR", "SIL_PNR"}, default "default" + The accuracy measure to return. + + ``default`` + The original accuracy measure already contained in the emgfile is + returned without any computation. + ``SIL`` + The Silhouette score is computed. + ``PNR`` + The pulse to noise ratio is computed. + ``SIL_PNR`` + Both the Silhouette score and the pulse to noise ratio are + computed. mvc : float, default 0 The maximum voluntary contraction (MVC). It is suggest to report MVC in Newton (N). If 0 (default), the user will be asked to imput it @@ -474,7 +490,7 @@ def basic_mus_properties( - compute_dr : calculate the discharge rate. - compute_covisi : calculate the coefficient of variation of interspike interval. - - compute_drvariability : claculate the DR variability. + - compute_drvariability : calculate the DR variability. Examples -------- @@ -509,6 +525,7 @@ def basic_mus_properties( 2 NaN 3 80.757524 87.150011 10.274494 11.087788 6.101529 4.789000 7.293547 5.846093 7.589531 8.055731 36.996894 35.308650 NaN 3 NaN 4 34.606886 37.569257 4.402912 4.779804 6.345692 5.333535 13.289651 9.694317 11.613640 11.109796 26.028689 29.372524 NaN """ + # TODO make new examples, also with accuracy # Check if we need to select the steady-state phase if (start_steady < 0 and end_steady < 0) or (start_steady < 0 or end_steady < 0): @@ -523,7 +540,7 @@ def basic_mus_properties( # First: create a dataframe that contains all the output exportable_df = [] - # Second: add basic information (MVC, MU number, PNR/SIL, Average PNR/SIL) + # Second: add basic information (MVC, MU number, ACCURACY, Average ACCURACY) if mvc == 0: # Ask the user to input MVC mvc = float( @@ -543,40 +560,94 @@ def basic_mus_properties( toappend = pd.DataFrame(toappend) exportable_df = pd.concat([exportable_df, toappend], axis=1) - # Calculate PNR - # Repeat the task for every new column to fill and concatenate - toappend = [] - for mu in range(emgfile["NUMBER_OF_MUS"]): - pnr = compute_pnr( - ipts=emgfile["IPTS"][mu], - mupulses=emgfile["MUPULSES"][mu], - fsamp=emgfile["FSAMP"], - ) - toappend.append({"PNR": pnr}) - toappend = pd.DataFrame(toappend) - exportable_df = pd.concat([exportable_df, toappend], axis=1) + if accuracy == "default": + # Report the original accuracy + toappend = emgfile["ACCURACY"] + toappend.columns = ["Accuracy"] + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + # Calculate avrage accuracy + avg_accuracy = exportable_df["Accuracy"].mean() + toappend = pd.DataFrame([{"avg_Accuracy": avg_accuracy}]) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + elif accuracy == "SIL": + # Calculate SIL + toappend = [] + for mu in range(emgfile["NUMBER_OF_MUS"]): + sil = compute_sil( + ipts=emgfile["IPTS"][mu], + mupulses=emgfile["MUPULSES"][mu], + ) + toappend.append({"SIL": sil}) + toappend = pd.DataFrame(toappend) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + # Calculate avrage SIL + avg_sil = exportable_df["SIL"].mean() + toappend = pd.DataFrame([{"avg_SIL": avg_sil}]) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + elif accuracy == "PNR": + # Calculate PNR + # Repeat the task for every new column to fill and concatenate + toappend = [] + for mu in range(emgfile["NUMBER_OF_MUS"]): + pnr = compute_pnr( + ipts=emgfile["IPTS"][mu], + mupulses=emgfile["MUPULSES"][mu], + fsamp=emgfile["FSAMP"], + ) + toappend.append({"PNR": pnr}) + toappend = pd.DataFrame(toappend) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + # Calculate avrage PNR + # dropna to avoid nan average. + avg_pnr = exportable_df["PNR"].mean() + toappend = pd.DataFrame([{"avg_PNR": avg_pnr}]) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + elif accuracy == "SIL_PNR": + # Calculate SIL + toappend = [] + for mu in range(emgfile["NUMBER_OF_MUS"]): + sil = compute_sil( + ipts=emgfile["IPTS"][mu], + mupulses=emgfile["MUPULSES"][mu], + ) + toappend.append({"SIL": sil}) + toappend = pd.DataFrame(toappend) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + # Calculate avrage SIL + avg_sil = exportable_df["SIL"].mean() + toappend = pd.DataFrame([{"avg_SIL": avg_sil}]) + exportable_df = pd.concat([exportable_df, toappend], axis=1) + + # Calculate PNR + # Repeat the task for every new column to fill and concatenate + toappend = [] + for mu in range(emgfile["NUMBER_OF_MUS"]): + pnr = compute_pnr( + ipts=emgfile["IPTS"][mu], + mupulses=emgfile["MUPULSES"][mu], + fsamp=emgfile["FSAMP"], + ) + toappend.append({"PNR": pnr}) + toappend = pd.DataFrame(toappend) + exportable_df = pd.concat([exportable_df, toappend], axis=1) - # Calculate avrage PNR - # dropna to avoid nan average. - avg_pnr = exportable_df["PNR"].mean() - toappend = pd.DataFrame([{"avg_PNR": avg_pnr}]) - exportable_df = pd.concat([exportable_df, toappend], axis=1) + # Calculate avrage PNR + # dropna to avoid nan average. + avg_pnr = exportable_df["PNR"].mean() + toappend = pd.DataFrame([{"avg_PNR": avg_pnr}]) + exportable_df = pd.concat([exportable_df, toappend], axis=1) - # Calculate SIL - toappend = [] - for mu in range(emgfile["NUMBER_OF_MUS"]): - sil = compute_sil( - ipts=emgfile["IPTS"][mu], - mupulses=emgfile["MUPULSES"][mu], + else: + raise ValueError( + f"accuracy must be one of 'default', 'SIL', 'PNR', 'SIL_PNR'. {accuracy} was passed instead" ) - toappend.append({"SIL": sil}) - toappend = pd.DataFrame(toappend) - exportable_df = pd.concat([exportable_df, toappend], axis=1) - - # Calculate avrage SIL - avg_sil = exportable_df["SIL"].mean() - toappend = pd.DataFrame([{"avg_SIL": avg_sil}]) - exportable_df = pd.concat([exportable_df, toappend], axis=1) # Calculate RT and DERT mus_thresholds = compute_thresholds(emgfile=emgfile, mvc=mvc) @@ -624,7 +695,7 @@ def compute_covisi( single_mu_number=-1, ): """ - Calculate theCOVisi. + Calculate the COVisi. This function calculates the coefficient of variation of interspike interval (COVisi). @@ -671,7 +742,7 @@ def compute_covisi( - compute_dr : calculate the discharge rate. - basic_mus_properties : calculate basic MUs properties on a trapezoidal contraction. - - compute_drvariability : claculate the DR variability. + - compute_drvariability : calculate the DR variability. Notes ----- @@ -832,7 +903,7 @@ def compute_drvariability( event_="rec_derec_steady", ): """ - Claculate the DR variability. + Calculate the DR variability. This function calculates the variability (as the coefficient of variation) of the instantaneous discharge rate (DR) at recruitment, derecruitment, diff --git a/openhdemg/library/electrodes.py b/openhdemg/library/electrodes.py index fec4bcb..97eddc7 100644 --- a/openhdemg/library/electrodes.py +++ b/openhdemg/library/electrodes.py @@ -110,6 +110,7 @@ def sort_rawemg( Sort RAW_SIGNAL based on matrix type and orientation. To date, built-in sorting functions have been implemented for the matrices: + Code (Orientation) GR08MM1305 (0, 180), GR04MM1305 (0, 180), @@ -129,10 +130,14 @@ def sort_rawemg( the ground (depending on the limb). dividebycolumn = bool, default True Whether to return the sorted channels classified by matrix column. - n_rows, n_cols : None or int, default None - The number of rows and columns of the matrix. This parameter is used to - divide the channels based on the matrix shape. These are normally - inferred by the matrix code and must be specified only if code == None. + n_rows : None or int, default None + The number of rows of the matrix. This parameter is used to divide the + channels based on the matrix shape. These are normally inferred by the + matrix code and must be specified only if code == None. + n_cols : None or int, default None + The number of columns of the matrix. This parameter is used to divide + the channels based on the matrix shape. These are normally inferred by + the matrix code and must be specified only if code == None. Returns ------- diff --git a/openhdemg/library/info.py b/openhdemg/library/info.py index 3f51a66..f4528c8 100644 --- a/openhdemg/library/info.py +++ b/openhdemg/library/info.py @@ -60,7 +60,7 @@ def data(self, emgfile): emgfile type is: emgfile keys are: - dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'PNR', 'SIL', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING']) + dict_keys(['SOURCE', 'FILENAME', 'RAW_SIGNAL', 'REF_SIGNAL', 'ACCURACY', 'IPTS', 'MUPULSES', 'FSAMP', 'IED', 'EMG_LENGTH', 'NUMBER_OF_MUS', 'BINARY_MUS_FIRING', 'EXTRAS']) Any key can be acced as emgfile[key]. emgfile['SOURCE'] is a of value: DEMUSE @@ -70,8 +70,8 @@ def data(self, emgfile): """ if emgfile["SOURCE"] in ["DEMUSE", "OTB", "CUSTOM"]: - print("\nData structure of the emgfile loaded with the function emg_from_otb.") - print("--------------------------------------------------------------------\n") + print("\nData structure of the emgfile") + print("-----------------------------\n") print(f"emgfile type is:\n{type(emgfile)}\n") print(f"emgfile keys are:\n{emgfile.keys()}\n") print("Any key can be acced as emgfile[key].\n") @@ -80,8 +80,7 @@ def data(self, emgfile): print("MUST NOTE: emgfile from OTB has 64 channels, from DEMUSE 65 (includes empty channel).") print(f"emgfile['RAW_SIGNAL'] is a {type(emgfile['RAW_SIGNAL'])} of value:\n{emgfile['RAW_SIGNAL']}\n") print(f"emgfile['REF_SIGNAL'] is a {type(emgfile['REF_SIGNAL'])} of value:\n{emgfile['REF_SIGNAL']}\n") - print(f"emgfile['PNR'] is a {type(emgfile['PNR'])} of value:\n{emgfile['PNR']}\n") - print(f"emgfile['SIL'] is a {type(emgfile['SIL'])} of value:\n{emgfile['SIL']}\n") + print(f"emgfile['ACCURACY'] is a {type(emgfile['ACCURACY'])} of value:\n{emgfile['ACCURACY']}\n") print(f"emgfile['IPTS'] is a {type(emgfile['IPTS'])} of value:\n{emgfile['IPTS']}\n") print(f"emgfile['MUPULSES'] is a {type(emgfile['MUPULSES'])} of length depending on total MUs number.") if emgfile['NUMBER_OF_MUS'] > 0: # Manage exceptions @@ -92,10 +91,11 @@ def data(self, emgfile): print(f"emgfile['EMG_LENGTH'] is a {type(emgfile['EMG_LENGTH'])} of value:\n{emgfile['EMG_LENGTH']}\n") print(f"emgfile['NUMBER_OF_MUS'] is a {type(emgfile['NUMBER_OF_MUS'])} of value:\n{emgfile['NUMBER_OF_MUS']}\n") print(f"emgfile['BINARY_MUS_FIRING'] is a {type(emgfile['BINARY_MUS_FIRING'])} of value:\n{emgfile['BINARY_MUS_FIRING']}\n") + print(f"emgfile['EXTRAS'] is a {type(emgfile['EXTRAS'])} of value:\n{emgfile['EXTRAS']}\n") - elif emgfile["SOURCE"] == "OTB_REFSIG": - print("\nData structure of the emgfile loaded with the function refsig_from_otb.") - print("-----------------------------------------------------------------------\n") + elif emgfile["SOURCE"] in ["OTB_REFSIG", "CUSTOMCSV_REFSIG"]: + print("\nData structure of the emgfile") + print("-----------------------------\n") print(f"emgfile type is:\n{type(emgfile)}\n") print(f"emgfile keys are:\n{emgfile.keys()}\n") print("Any key can be acced as emgfile[key].\n") @@ -103,6 +103,7 @@ def data(self, emgfile): print(f"emgfile['FILENAME'] is a {type(emgfile['FILENAME'])} of value:\n{emgfile['FILENAME']}\n") print(f"emgfile['FSAMP'] is a {type(emgfile['FSAMP'])} of value:\n{emgfile['FSAMP']}\n") print(f"emgfile['REF_SIGNAL'] is a {type(emgfile['REF_SIGNAL'])} of value:\n{emgfile['REF_SIGNAL']}\n") + print(f"emgfile['EXTRAS'] is a {type(emgfile['EXTRAS'])} of value:\n{emgfile['EXTRAS']}\n") else: raise ValueError(f"Source '{emgfile['SOURCE']}' not recognised") @@ -217,7 +218,7 @@ def aboutus(self): Us -- - People that contributed to the development of this project are: + The developers of this project are: Mr. Giacomo Valli: The creator of the project and the developer of the library. @@ -242,6 +243,9 @@ def aboutus(self): methods to evaluate muscle morphological as well architectural parameters. \x1B[0m + + For the full list of contributors visit: + https://www.giacomovalli.com/openhdemg/about-us/ """ # Make Text Bold and Italic with Escape Sequence @@ -264,15 +268,17 @@ def contacts(self): -------- >>> import openhdemg.library as emg >>> emg.info().contacts() - "Name": "Giacomo Valli", - "Email": "giacomo.valli@phd.unipd.it", - "Twitter": "@giacomo_valli" + "Primary contact": "openhdemg@gmail.com", + "Twitter": "@openhdemg", + "Maintainer": "Giacomo Valli", + "Maintainer Email": "giacomo.valli@phd.unipd.it", """ contact = { - "Name": "Giacomo Valli", - "Email": "giacomo.valli@phd.unipd.it", + "Primary contact": "openhdemg@gmail.com", "Twitter": "@openhdemg", + "Maintainer": "Giacomo Valli", + "Maintainer Email": "giacomo.valli@phd.unipd.it", } # Pretty dict printing @@ -284,12 +290,17 @@ def contacts(self): def links(self): """ Print a collection of useful links. + + Examples + -------- + >>> import openhdemg.library as emg + >>> emg.info().links() """ links = { "Project Website": "https://www.giacomovalli.com/openhdemg/", - "Release Notes": "https://www.giacomovalli.com/openhdemg/What%27s-New/", - "Cite Us": "https://www.giacomovalli.com/openhdemg/Cite-Us/", + "Release Notes": "https://www.giacomovalli.com/openhdemg/what%27s-new/", + "Cite Us": "https://www.giacomovalli.com/openhdemg/cite-us/", "Discussion Forum": "https://github.com/GiacomoValliPhD/openhdemg/discussions", "Report Bugs": "https://github.com/GiacomoValliPhD/openhdemg/issues", } @@ -303,6 +314,11 @@ def links(self): def citeus(self): """ Print how to cite the project. + + Examples + -------- + >>> import openhdemg.library as emg + >>> emg.info().citeus() """ cite = { diff --git a/openhdemg/library/mathtools.py b/openhdemg/library/mathtools.py index 39114de..5360887 100644 --- a/openhdemg/library/mathtools.py +++ b/openhdemg/library/mathtools.py @@ -168,7 +168,7 @@ def norm_twod_xcorr(df1, df2, mode="full"): 1. Load the EMG file and band-pass filter the raw EMG signal 2. Sort the matrix channels and compute the spike-triggered average 3. Extract the STA of the MUs of interest from all the STAs - 4. Unpack the STAs of single MUs and remove np.nan to pas them to + 4. Unpack the STAs of single MUs and remove np.nan to pass them to norm_twod_xcorr 5. Compute 2dxcorr to identify a common lag/delay diff --git a/openhdemg/library/muap.py b/openhdemg/library/muap.py index ef9a2a5..aedd7e2 100644 --- a/openhdemg/library/muap.py +++ b/openhdemg/library/muap.py @@ -226,9 +226,10 @@ def sta( Every key of the dictionary represents a different column of the matrix. Rows are stored in the dict as a pd.DataFrame. firings : list or str {"all"}, default [0, 50] - The range of firnings to be used for the STA. + The range of firings to be used for the STA. If a MU has less firings than the range, the upper limit is adjusted accordingly. + ``all`` The STA is calculated over all the firings. timewindow : int, default 50 @@ -369,6 +370,7 @@ def parallel(mu): sorted_rawemg_sta["munumber"] = mu return sorted_rawemg_sta + # TODO verify built-in options to return from joblib.Parallel # Start parallel execution # Meausere running time @@ -411,10 +413,10 @@ def st_muap(emgfile, sorted_rawemg, timewindow=50): ------- stmuap : dict dict containing a dict of ST MUAPs (pd.DataFrame) for every MUs. - pd.DataFrames containing the ST MUAPs are organised based on matrix + The pd.DataFrames containing the ST MUAPs are organised based on matrix rows (dict) and matrix channel. For example, the ST MUAPs of the first MU (0), in the second electrode - of the matrix can be accessed as stmuap[0]["col0"][1]. + of the first matrix column can be accessed as stmuap[0]["col0"][1]. See also -------- @@ -623,19 +625,17 @@ def align_by_xcorr(sta_mu1, sta_mu2, finalduration=0.5): Returns ------- aligned_sta1 : dict - A dictionary containing the aligned and STA of the first MU - with the final expected timewindow - (duration of sta_mu1 * finalduration). + A dictionary containing the aligned STA of the first MU with the final + expected timewindow (duration of sta_mu * finalduration). aligned_sta2 : dict - A dictionary containing the aligned and STA of the second MU - with the final expected timewindow - (duration of sta_mu1 * finalduration). + A dictionary containing the aligned STA of the second MU with the + final expected timewindow (duration of sta_mu * finalduration). See also -------- - sta : computes the STA of every MUs. - norm_twod_xcorr : normalised 2-dimensional cross-correlation of STAs of - two MUS. + two MUs. Notes ----- @@ -764,7 +764,7 @@ def tracking( emgfile2 : dict The dictionary containing the second emgfile. firings : list or str {"all"}, default "all" - The range of firnings to be used for the STA. + The range of firings to be used for the STA. If a MU has less firings than the range, the upper limit is adjusted accordingly. ``all`` @@ -786,10 +786,14 @@ def tracking( orientation : int {0, 180}, default 180 Orientation in degree of the matrix (same as in OTBiolab). E.g. 180 corresponds to the matrix connection toward the user. - n_rows, n_cols : None or int, default None - The number of rows and columns of the matrix. This parameter is used to - divide the channels based on the matrix shape. These are normally - inferred by the matrix code and must be specified only if code == None. + n_rows : None or int, default None + The number of rows of the matrix. This parameter is used to divide the + channels based on the matrix shape. These are normally inferred by the + matrix code and must be specified only if code == None. + n_cols : None or int, default None + The number of columns of the matrix. This parameter is used to divide + the channels based on the matrix shape. These are normally inferred by + the matrix code and must be specified only if code == None. exclude_belowthreshold : bool, default True Whether to exclude results with XCC below threshold. filter : bool, default True @@ -813,7 +817,7 @@ def tracking( -------- - sta : computes the STA of every MUs. - norm_twod_xcorr : normalised 2-dimensional cross-correlation of STAs of - two MUS. + two MUs. - remove_duplicates_between : remove duplicated MUs across two different files based on STA. @@ -1055,7 +1059,7 @@ def remove_duplicates_between( emgfile2 : dict The dictionary containing the second emgfile. firings : list or str {"all"}, default "all" - The range of firnings to be used for the STA. + The range of firings to be used for the STA. If a MU has less firings than the range, the upper limit is adjusted accordingly. ``all`` @@ -1077,24 +1081,26 @@ def remove_duplicates_between( orientation : int {0, 180}, default 180 Orientation in degree of the matrix (same as in OTBiolab). E.g. 180 corresponds to the matrix connection toward the user. - n_rows, n_cols : None or int, default None - The number of rows and columns of the matrix. This parameter is used to - divide the channels based on the matrix shape. These are normally - inferred by the matrix code and must be specified only if code == None. + n_rows : None or int, default None + The number of rows of the matrix. This parameter is used to divide the + channels based on the matrix shape. These are normally inferred by the + matrix code and must be specified only if code == None. + n_cols : None or int, default None + The number of columns of the matrix. This parameter is used to divide + the channels based on the matrix shape. These are normally inferred by + the matrix code and must be specified only if code == None. filter : bool, default True If true, when the same MU has a match of XCC > threshold with multiple MUs, only the match with the highest XCC is returned. show : bool, default False Whether to plot the STA of pairs of MUs with XCC above threshold. - which : str {"munumber", "PNR", "SIL"} + which : str {"munumber", "accuracy"} How to remove the duplicated MUs. ``munumber`` Duplicated MUs are removed from the file with more MUs. - ``SIL`` - The MU with the lowest SIL is removed. - ``PNR`` - The MU with the lowest PNR is removed. + ``accuracy`` + The MU with the lowest accuracy is removed. Returns ------- @@ -1108,7 +1114,7 @@ def remove_duplicates_between( -------- - sta : computes the STA of every MUs. - norm_twod_xcorr : normalised 2-dimensional cross-correlation of STAs of - two MUS. + two MUs. - tracking : track MUs across two different files. Examples @@ -1117,6 +1123,7 @@ def remove_duplicates_between( without duplicates. The duplicates are removed from the file with more MUs. + >>> import openhdemg.library as emg >>> emgfile1 = emg.askopenfile(filesource="OTB", otb_ext_factor=8) >>> emgfile2 = emg.askopenfile(filesource="OTB", otb_ext_factor=8) >>> emgfile1, emgfile2, tracking_res = emg.remove_duplicates_between( @@ -1180,42 +1187,16 @@ def remove_duplicates_between( return emgfile1, emgfile2, tracking_res - elif which == "PNR": + elif which == "accuracy": # Create a list containing which MU to remove in which file based - # on PNR value. + # on ACCURACY value. to_remove1 = [] to_remove2 = [] for i, row in tracking_res.iterrows(): - pnr1 = emgfile1["PNR"].loc[int(row["MU_file1"])] - pnr2 = emgfile2["PNR"].loc[int(row["MU_file2"])] - - if pnr1[0] <= pnr2[0]: - # This MU should be removed from emgfile1 - to_remove1.append(int(row["MU_file1"])) - else: - # This MU should be removed from emgfile2 - to_remove2.append(int(row["MU_file2"])) - - # Delete the MUs - emgfile1 = delete_mus( - emgfile=emgfile1, munumber=to_remove1, if_single_mu="remove" - ) - emgfile2 = delete_mus( - emgfile=emgfile2, munumber=to_remove2, if_single_mu="remove" - ) - - return emgfile1, emgfile2, tracking_res - - elif which == "SIL": - # Create a list containing which MU to remove in which file based - # on SIL score. - to_remove1 = [] - to_remove2 = [] - for _, row in tracking_res.iterrows(): - sil1 = emgfile1["SIL"].loc[int(row["MU_file1"])] - sil2 = emgfile2["SIL"].loc[int(row["MU_file2"])] + acc1 = emgfile1["ACCURACY"].loc[int(row["MU_file1"])] + acc2 = emgfile2["ACCURACY"].loc[int(row["MU_file2"])] - if sil1[0] <= sil2[0]: + if acc1[0] <= acc2[0]: # This MU should be removed from emgfile1 to_remove1.append(int(row["MU_file1"])) else: @@ -1234,7 +1215,7 @@ def remove_duplicates_between( else: raise ValueError( - f"which can be one of 'munumber', 'PNR', 'SIL'. {which} was passed instead" + f"which can be one of 'munumber' or 'accuracy'. {which} was passed instead" ) @@ -1326,7 +1307,7 @@ class MUcv_gui: matrix. Rows are stored in the dict as a pd.DataFrame. n_firings : list or str {"all"}, default [0, 50] - The range of firnings to be used for the STA. + The range of firings to be used for the STA. If a MU has less firings than the range, the upper limit is adjusted accordingly. ``all`` diff --git a/openhdemg/library/openfiles.py b/openhdemg/library/openfiles.py index 0524985..f5e124c 100644 --- a/openhdemg/library/openfiles.py +++ b/openhdemg/library/openfiles.py @@ -20,11 +20,11 @@ order to be compatible with this library should be exported with a strict structure as described in the function emg_from_otb. In both cases, the input file is a .mat file. -refsig_from_otb : - Used to load files from the OTBiolab+ software that contain only - the REF_SIGNAL. emg_from_customcsv : Used to load custom file formats contained in .csv files. +refsig_from_otb and refsig_from_customcsv: + Used to load files from the OTBiolab+ software or from a custom .csv file + that contain only the REF_SIGNAL. save_json_emgfile, emg_from_json : Used to save the working file to a .json file or to load the .json file. @@ -35,11 +35,11 @@ Notes ----- Once opened, the file is returned as a dict with keys: - "SOURCE" : source of the file (i.e., "DEMUSE", "OTB", "CUSTOM") + "SOURCE" : source of the file (i.e., "CUSTOMCSV", "DEMUSE", "OTB") + "FILENAME" : the name of the opened file "RAW_SIGNAL" : the raw EMG signal "REF_SIGNAL" : the reference signal - "PNR" : pulse to noise ratio - "SIL" : silouette score + "ACCURACY" : accuracy score (depending on source file type) "IPTS" : pulse train (decomposed source) "MUPULSES" : instants of firing "FSAMP" : sampling frequency @@ -47,11 +47,14 @@ "EMG_LENGTH" : length of the emg file (in samples) "NUMBER_OF_MUS" : total number of MUs "BINARY_MUS_FIRING" : binary representation of MUs firings + "EXTRAS" : additional custom values -The only exception is when OTB files are loaded with just the reference signal: - "SOURCE": source of the file (i.e., "OTB_REFSIG") +The only exception is when files are loaded with just the reference signal: + "SOURCE": source of the file (i.e., "CUSTOMCSV_REFSIG", "OTB_REFSIG") + "FILENAME" : the name of the opened file "FSAMP": sampling frequency "REF_SIGNAL": the reference signal + "EXTRAS" : additional custom values Additional informations can be found in the info module (emg.info()) and in the function's description. @@ -65,20 +68,21 @@ # emg_from_demuse, # refsig_from_otb, # emg_from_customcsv, +# refsig_from_customcsv # save_json_emgfile, # emg_from_json, # askopenfile, # asksavefile, # emg_from_samplefile, -# ) +# ) # TODO add emg_from_delsys here, in init, in upper description and in docs description from scipy.io import loadmat import pandas as pd import numpy as np from openhdemg.library.electrodes import * -from openhdemg.library.mathtools import compute_pnr, compute_sil -from openhdemg.library.tools import create_binary_firings +from openhdemg.library.mathtools import compute_sil +from openhdemg.library.tools import create_binary_firings, mupulses_from_binary from tkinter import * from tkinter import filedialog import json @@ -87,12 +91,12 @@ import os -# --------------------------------------------------------------------- +# --------------------------------------------------------------------- # # Main function to open decomposed files coming from DEMUSE. def emg_from_demuse(filepath): """ - Import the .mat file used in DEMUSE. + Import the .mat file decomposed in DEMUSE. Parameters ---------- @@ -108,7 +112,7 @@ def emg_from_demuse(filepath): See also -------- - - emg_from_otb : import the .mat file exportable by OTBiolab+. + - emg_from_otb : import the decomposed .mat file exportable by OTBiolab+. - refsig_from_otb : import REF_SIGNAL in the .mat file exportable by OTBiolab+. - emg_from_customcsv : Import custom data from a .csv file. @@ -121,13 +125,13 @@ def emg_from_demuse(filepath): (as for OTB matrix standards) in the case of a 64 electrodes matrix. Structure of the emgfile: + emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL + "ACCURACY": SIL "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -135,8 +139,12 @@ def emg_from_demuse(filepath): "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, } + For DEMUSE files, the accuracy is estimated with the silhouette (SIL) + score. + Examples -------- For an extended explanation of the imported emgfile: @@ -147,10 +155,10 @@ def emg_from_demuse(filepath): >>> info.data(emgfile) """ + # Load the .mat file mat_file = loadmat(filepath, simplify_cells=True) # Parse .mat obtained from DEMUSE to see the available variables - # First: see the variables name """ print( "\n--------------------------------\nAvailable dict keys are:\n\n{}\n".format( @@ -159,29 +167,28 @@ def emg_from_demuse(filepath): ) """ - # Second: collect the necessary variables in a pd.DataFrame (df) - # or list (for matlab cell arrays) - - # Collect the REF_SIGNAL - if "ref_signal" in mat_file.keys(): + # First, get the basic information and compulsory variables (i.e., + # RAW_SIGNAL, IPTS, MUPULSES, BINARY_MUS_FIRING) in a pd.DataFrame (df) or + # list (for matlab cell arrays). - # Catch the case for float values that cannot be directly added to a - # dataframe - if isinstance(mat_file["ref_signal"], float): - res = {0: mat_file["ref_signal"]} - REF_SIGNAL = pd.DataFrame(res, index=[0]) + # Use this to know the data source and name of the file + SOURCE = "DEMUSE" + FILENAME = os.path.basename(filepath) + FSAMP = float(mat_file["fsamp"]) + IED = float(mat_file["IED"]) - else: - REF_SIGNAL = pd.DataFrame(mat_file["ref_signal"]) + # Get RAW_SIGNAL + if "SIG" in mat_file.keys(): + mat = mat_file["SIG"].ravel(order="F") + # "F" means to index the elements in column-major + RAW_SIGNAL = pd.DataFrame(list(map(np.ravel, mat))).transpose() else: - warnings.warn( - "\nVariable ref_signal was not found in the mat file, check the spelling against the dict_keys\n" + raise ValueError( + "\nVariable 'SIG' not found in the .mat file\n" ) - REF_SIGNAL = np.nan - - # Collect the IPTS + # Get IPTS if "IPTs" in mat_file.keys(): # Catch the exception of a single MU that would create an alrerady # transposed pd.DataFrame @@ -192,90 +199,72 @@ def emg_from_demuse(filepath): IPTS = pd.DataFrame(mat_file["IPTs"]).transpose() else: - warnings.warn( - "\nVariable IPTs was not found in the mat file, check the spelling against the dict_keys\n" + raise ValueError( + "\nVariable 'IPTS' not found in the .mat file\n" ) - IPTS = np.nan - - # Collect Sampling frequency, Interelectrode distance, - # File length and number of MUs - FSAMP = int(mat_file["fsamp"]) - IED = int(mat_file["IED"]) + # Get EMG_LENGTH and NUMBER_OF_MUS EMG_LENGTH, NUMBER_OF_MUS = IPTS.shape - # Collect the MUPULSES, subtract 1 to MUPULSES because these are values in - # base 1 (MATLAB) and manage exception of single MU. + # Get MUPULSES/BINARY_MUS_FIRING + # Subtract 1 to MUPULSES because these are values in base 1 (MATLAB) and + # manage exception of single MU thah would create a list and not a list of + # arrays. if "MUPulses" in mat_file.keys(): MUPULSES = list(mat_file["MUPulses"]) + for pos, pulses in enumerate(MUPULSES): + MUPULSES[pos] = pulses - 1 + + if NUMBER_OF_MUS == 1: + MUPULSES = [np.array(MUPULSES)] else: - warnings.warn( - "\nVariable MUPulses was not found in the mat file, check the spelling against the dict_keys\n" + raise ValueError( + "\nVariable 'MUPulses' not found in the .mat file\n" ) - MUPULSES = np.nan - - for pos, pulses in enumerate(MUPULSES): - MUPULSES[pos] = pulses - 1 - - if NUMBER_OF_MUS == 1: - MUPULSES = [np.array(MUPULSES)] - - # Collect firing times + # Calculate BINARY_MUS_FIRING BINARY_MUS_FIRING = create_binary_firings( emg_length=EMG_LENGTH, number_of_mus=NUMBER_OF_MUS, mupulses=MUPULSES, ) - # Collect the raw EMG signal - if "SIG" in mat_file.keys(): - mat = mat_file["SIG"].ravel(order="F") - # "F" means to index the elements in column-major - RAW_SIGNAL = pd.DataFrame(list(map(np.ravel, mat))).transpose() + # Second, get/generate the other variables + # Get REF_SIGNAL + if "ref_signal" in mat_file.keys(): + # Catch the case for float values that cannot be directly added to a + # dataframe + if isinstance(mat_file["ref_signal"], float): + res = {0: mat_file["ref_signal"]} + REF_SIGNAL = pd.DataFrame(res, index=[0]) + + else: + REF_SIGNAL = pd.DataFrame(mat_file["ref_signal"]) else: + REF_SIGNAL = pd.DataFrame(columns=[0]) warnings.warn( - "\nVariable SIG was not found in the mat file, check the spelling against the dict_keys\n" + "\nVariable ref_signal not found in the .mat file, it might be necessary for some analyses\n" ) - RAW_SIGNAL = np.nan - - # Use this to know the data source and name of the file - SOURCE = "DEMUSE" - FILENAME = os.path.basename(filepath) - + # Estimate ACCURACY (SIL) if NUMBER_OF_MUS > 0: - # Calculate the PNR - to_append = [] - for mu in range(NUMBER_OF_MUS): - pnr = compute_pnr( - ipts=IPTS[mu], - mupulses=MUPULSES[mu], - fsamp=FSAMP, - ) - to_append.append(pnr) - PNR = pd.DataFrame(to_append) - - # Calculate the SIL to_append = [] for mu in range(NUMBER_OF_MUS): sil = compute_sil(ipts=IPTS[mu], mupulses=MUPULSES[mu]) to_append.append(sil) - SIL = pd.DataFrame(to_append) + ACCURACY = pd.DataFrame(to_append) else: - PNR = np.nan - SIL = np.nan + ACCURACY = pd.DataFrame(columns=[0]) emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL, + "ACCURACY": ACCURACY, "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -283,6 +272,7 @@ def emg_from_demuse(filepath): "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": pd.DataFrame(columns=[0]), } return emgfile @@ -317,11 +307,11 @@ def get_otb_refsignal(df, refsig): assert refsig[0] in [ True, False, - ], f"refsig[0] must be true or false. {refsig[0]} was passed instead." + ], f"refsig[0] must be 'true' or 'false'. {refsig[0]} was passed instead." assert refsig[1] in [ "fullsampled", "subsampled", - ], f"refsig[1] must be fullsampled or subsampled. {refsig[1]} was passed instead." + ], f"refsig[1] must be 'fullsampled' or 'subsampled'. {refsig[1]} was passed instead." if refsig[0] is True: if refsig[1] == "subsampled": @@ -336,17 +326,17 @@ def get_otb_refsignal(df, refsig): # REF_SIGNAL is expected to be expressed as % of the MVC if max(REF_SIGNAL_SUBSAMPLED[0]) > 100: warnings.warn( - "\nALERT! Ref signal grater than 100, did you use values normalised to the MVC?\n" + "\nALERT! Ref signal greater than 100, did you use values normalised to the MVC?\n" ) return REF_SIGNAL_SUBSAMPLED else: warnings.warn( - "\nReference signal not found, it might be necessary for some analysis\n" + "\nReference signal not found, it might be necessary for some analyses\n" ) - return np.nan + return pd.DataFrame(columns=[0]) elif refsig[1] == "fullsampled": # Extract the acquired path (raw data) @@ -366,15 +356,15 @@ def get_otb_refsignal(df, refsig): else: warnings.warn( - "\nReference signal not found, it might be necessary for some analysis\n" + "\nReference signal not found, it might be necessary for some analyses\n" ) - return np.nan + return pd.DataFrame(columns=[0]) else: - warnings.warn("\nNot searched for reference signal, it might be necessary for some analysis\n") + warnings.warn("\nNot searched for reference signal, it might be necessary for some analyses\n") - return np.nan + return pd.DataFrame(columns=[0]) def get_otb_decomposition(df): @@ -399,50 +389,22 @@ def get_otb_decomposition(df): IPTS.columns = np.arange(len(IPTS.columns)) # Verify to have the IPTS if IPTS.empty: - IPTS = np.nan + raise ValueError( + "\nSource for decomposition (IPTS) not found in the .mat file\n" + ) # Extract the BINARY_MUS_FIRING and rename columns progressively BINARY_MUS_FIRING = df.filter(regex="Decomposition of") BINARY_MUS_FIRING.columns = np.arange(len(BINARY_MUS_FIRING.columns)) # Verify to have the BINARY_MUS_FIRING if BINARY_MUS_FIRING.empty: - BINARY_MUS_FIRING = np.nan + raise ValueError( + "\nDecomposition of (BINARY_MUS_FIRING) not found in the .mat file\n" + ) return IPTS, BINARY_MUS_FIRING -def get_otb_mupulses(binarymusfiring): - """ - Extract the MUPULSES from the OTB .mat file. - - Parameters - ---------- - binarymusfiring : pd.DataFrame - A pd.DataFrame containing the binary representation of MUs firings. - - Returns - ------- - MUPULSES : list - A list of ndarrays containing the firing time of each MU. - """ - - # Create empty list of lists to fill with ndarrays containing the MUPULSES - # (point of firing) - numberofMUs = len(binarymusfiring.columns) - MUPULSES = [[] for _ in range(numberofMUs)] - - for i in binarymusfiring: # Loop all the MUs - my_ndarray = [] - for idx, x in binarymusfiring[i].items(): # Loop the MU firing times - if x > 0: - my_ndarray.append(idx) # Take the firing time and add it to the ndarray - - my_ndarray = np.array(my_ndarray) - MUPULSES[i] = my_ndarray - - return MUPULSES - - def get_otb_ied(df): """ Extract the IED from the OTB .mat file. @@ -455,7 +417,7 @@ def get_otb_ied(df): Returns ------- - IED : int + IED : float The interelectrode distance in millimeters. """ @@ -463,20 +425,29 @@ def get_otb_ied(df): # Check the matrix used in the columns name # (in the df obtained from OTBiolab+) if matrix in str(df.columns): - IED = int(OTBelectrodes_ied[matrix]) + IED = float(OTBelectrodes_ied[matrix]) return IED + # If no matrix is found and we exit the loop: + warnings.warn( + "OTB recording grid not found, IED could not be inferred" + ) + + return np.nan + -def get_otb_rawsignal(df): +def get_otb_rawsignal(df, extras_regex): """ - Extract the IED from the OTB .mat file. + Extract the raw signal from the OTB .mat file. Parameters ---------- df : pd.DataFrame A pd.DataFrame containing all the informations extracted from the OTB .mat file. + extras_regex : str + A regex pattern unequivocally identifying the EXTRAS. Returns ------- @@ -487,11 +458,17 @@ def get_otb_rawsignal(df): # Drop all the known columns different from the raw EMG signal. # This is a workaround since the OTBiolab+ software does not export a # unique name for the raw EMG signal. - pattern = "Source for decomposition|Decomposition of|acquired data|performed path" + base_pattern = "Source for decomposition|Decomposition of|acquired data|performed path" + if extras_regex is None: + pattern = base_pattern + else: + pattern = base_pattern + "|" + extras_regex + emg_df = df[df.columns.drop(list(df.filter(regex=pattern)))] # Check if the number of remaining columns matches the expected number of # matrix channels. + expectedchannels = np.nan for matrix in OTBelectrodes_Nelectrodes.keys(): # Check the matrix used in the columns name (in the emg_df) to know # the number of expected channels. @@ -499,6 +476,9 @@ def get_otb_rawsignal(df): expectedchannels = int(OTBelectrodes_Nelectrodes[matrix]) break + if expectedchannels is np.nan: + raise ValueError("Matrix not recognised") + if len(emg_df.columns) == expectedchannels: emg_df.columns = np.arange(len(emg_df.columns)) RAW_SIGNAL = emg_df @@ -508,17 +488,47 @@ def get_otb_rawsignal(df): else: # This check here is usefull to control that only the appropriate # elements have been included in the .mat file exported from OTBiolab+. - raise Exception( - "Failure in searching the raw signal, please check that it is present in the .mat file and that only the accepted parameters have been included" + raise ValueError( + "\nFailure in searching the raw signal, please check that it is present in the .mat file and that only the accepted parameters have been included\n" ) +def get_otb_extras(df, extras): + """ + Extract the EXTRAS from the OTB .mat file. + + Parameters + ---------- + df : pd.DataFrame + A pd.DataFrame containing all the informations extracted + from the OTB .mat file. + + Returns + ------- + EXTRAS : pd.DataFrame + A pd.DataFrame containing the EXTRAS. + """ + + if extras is None: + + return pd.DataFrame(columns=[0]) + + else: + EXTRAS = df.filter(regex=extras) + + return EXTRAS + + # --------------------------------------------------------------------- # Main function to open decomposed files coming from OTBiolab+. # This function calls the functions defined above def emg_from_otb( - filepath, ext_factor=8, refsig=[True, "fullsampled"], version="1.5.8.0" + filepath, + ext_factor=8, + refsig=[True, "fullsampled"], + version="1.5.9.3", + extras=None, ): """ Import the .mat file exportable by OTBiolab+. @@ -538,7 +548,7 @@ def emg_from_otb( Whether to seacrh also for the REF_SIGNAL and whether to load the full or sub-sampled one. The list is composed as [bool, str]. str can be "fullsampled" or "subsampled". Please read notes section. - version : str, default "1.5.8.0" + version : str, default "1.5.9.3" Version of the OTBiolab+ software used (4 points). Tested versions are: "1.5.3.0", @@ -548,9 +558,14 @@ def emg_from_otb( "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", If your specific version is not available in the tested versions, - trying with the closer one usually works, but please double check the - results. + trying with the closer one usually works. + extras : None or str, default None + Extras is used to store additional custom values. These information + will be stored in a pd.DataFrame with columns named as in the .mat + file. If not None, pass a regex pattern unequivocally identifying the + variable in the .mat file to load as extras. Returns ------- @@ -573,29 +588,31 @@ def emg_from_otb( --------- The returned file is called ``emgfile`` for convention. - The input .mat file exported from the OTBiolab+ software should have a + The input .mat file exported from the OTBiolab+ software must have a specific content: - - refsig signal is optional but, if present, there should be the + + - The reference signal is optional but, if present, there should be the fullsampled or the subsampled version (in OTBioLab+ the "performed path" refers to the subsampled signal, the "acquired data" to the fullsampled signal), REF_SIGNAL is expected to be expressed as % of the MVC (but not compulsory). - Both the IPTS ('Source for decomposition...' in OTBioLab+) and the - BINARY_MUS_FIRING ('Decomposition of...' in OTBioLab+) should be + BINARY_MUS_FIRING ('Decomposition of...' in OTBioLab+) must be present. - - The raw EMG signal should be present (it has no specific name in + - The raw EMG signal must be present (it has no specific name in OTBioLab+) with all the channels. Don't exclude unwanted channels before exporting the .mat file. - - NO OTHER ELEMENTS SHOULD BE PRESENT! + - NO OTHER ELEMENTS SHOULD BE PRESENT, unless an appropriate regex pattern + is passed to 'extras='! Structure of the returned emgfile: + emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL, + "ACCURACY": SIL, "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -603,8 +620,12 @@ def emg_from_otb( "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, } + For OTBiolab+ files, the accuracy is estimated with the silhouette (SIL) + score. + Examples -------- For an extended explanation of the imported emgfile use: @@ -617,7 +638,7 @@ def emg_from_otb( mat_file = loadmat(filepath, simplify_cells=True) - # Parse .mat obtained from DEMUSE to see the available variables + # Parse .mat obtained from OTBiolab+ to see the available variables """ print( "\n--------------------------------\nAvailable dict keys are:\n\n{}\n".format( mat_file.keys() @@ -633,9 +654,12 @@ def emg_from_otb( "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", ] if version not in valid_versions: - raise ValueError(f"Specified version is not valid. Use one of:\n{valid_versions}") + raise ValueError( + f"\nSpecified version is not valid. Use one of:\n{valid_versions}\n" + ) if version in [ "1.5.3.0", @@ -645,79 +669,91 @@ def emg_from_otb( "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", ]: # Simplify (rename) columns description and extract all the parameters # in a pd.DataFrame df = pd.DataFrame(mat_file["Data"], columns=mat_file["Description"]) - # Collect the REF_SIGNAL - REF_SIGNAL = get_otb_refsignal(df=df, refsig=refsig) + # First, get the basic information and compulsory variables (i.e., + # RAW_SIGNAL, IPTS, MUPULSES, BINARY_MUS_FIRING) in a pd.DataFrame (df) or + # list (for matlab cell arrays). - # Collect the IPTS and the firing times + # Use this to know the data source and name of the file + SOURCE = "OTB" + FILENAME = os.path.basename(filepath) + FSAMP = float(mat_file["SamplingFrequency"]) + IED = get_otb_ied(df=df) + + # Get RAW_SIGNAL + RAW_SIGNAL = get_otb_rawsignal(df=df, extras_regex=extras) + + # Get IPTS and BINARY_MUS_FIRING IPTS, BINARY_MUS_FIRING = get_otb_decomposition(df=df) # Align BINARY_MUS_FIRING to IPTS BINARY_MUS_FIRING = BINARY_MUS_FIRING.shift(- int(ext_factor)) BINARY_MUS_FIRING.fillna(value=0, inplace=True) - # Collect additional parameters + # Get MUPULSES + MUPULSES = mupulses_from_binary(binarymusfiring=BINARY_MUS_FIRING) + + # Get EMG_LENGTH and NUMBER_OF_MUS EMG_LENGTH, NUMBER_OF_MUS = IPTS.shape - MUPULSES = get_otb_mupulses(binarymusfiring=BINARY_MUS_FIRING) - FSAMP = int(mat_file["SamplingFrequency"]) - IED = get_otb_ied(df=df) - RAW_SIGNAL = get_otb_rawsignal(df) - # Use this to know the data source and name of the file - SOURCE = "OTB" - FILENAME = os.path.basename(filepath) + # Get REF_SIGNAL + REF_SIGNAL = get_otb_refsignal(df=df, refsig=refsig) + # Estimate ACCURACY (SIL) if NUMBER_OF_MUS > 0: - # Calculate the PNR - to_append = [] - for mu in range(NUMBER_OF_MUS): - pnr = compute_pnr(ipts=IPTS[mu], mupulses=MUPULSES[mu], fsamp=FSAMP) - to_append.append(pnr) - PNR = pd.DataFrame(to_append) - - # Calculate the SIL to_append = [] for mu in range(NUMBER_OF_MUS): sil = compute_sil(ipts=IPTS[mu], mupulses=MUPULSES[mu]) to_append.append(sil) - SIL = pd.DataFrame(to_append) + ACCURACY = pd.DataFrame(to_append) else: - PNR = np.nan - SIL = np.nan + ACCURACY = pd.DataFrame(columns=[0]) - emgfile = { - "SOURCE": SOURCE, - "FILENAME": FILENAME, - "RAW_SIGNAL": RAW_SIGNAL, - "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL, - "IPTS": IPTS, - "MUPULSES": MUPULSES, - "FSAMP": FSAMP, - "IED": IED, - "EMG_LENGTH": EMG_LENGTH, - "NUMBER_OF_MUS": NUMBER_OF_MUS, - "BINARY_MUS_FIRING": BINARY_MUS_FIRING, - } + # Get EXTRAS + EXTRAS = get_otb_extras(df=df, extras=extras) + + emgfile = { + "SOURCE": SOURCE, + "FILENAME": FILENAME, + "RAW_SIGNAL": RAW_SIGNAL, + "REF_SIGNAL": REF_SIGNAL, + "ACCURACY": ACCURACY, + "IPTS": IPTS, + "MUPULSES": MUPULSES, + "FSAMP": FSAMP, + "IED": IED, + "EMG_LENGTH": EMG_LENGTH, + "NUMBER_OF_MUS": NUMBER_OF_MUS, + "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, + } + + return emgfile - return emgfile +# --------------------------------------------------------------------- +# Function to load the reference signal from OBIolab+. -def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): +def refsig_from_otb( + filepath, + refsig="fullsampled", + version="1.5.9.3", + extras=None, +): """ - Import REF_SIGNAL in the .mat file exportable by OTBiolab+. + Import the reference signal in the .mat file exportable by OTBiolab+. This function is used to import the .mat file exportable by the OTBiolab+ software as a dictionary of Python objects (mainly pandas dataframes). Compared to the function emg_from_otb, this function only imports the REF_SIGNAL and, therefore, it can be used for special cases where only the - REF_SIGNAL is necessary. This will allow a faster execution of the script - and to avoid exceptions for missing data. + REF_SIGNAL is necessary. This will allow for a faster execution of the + script and to avoid exceptions for missing data. Parameters ---------- @@ -728,7 +764,7 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): refsig : str {"fullsampled", "subsampled"}, default "fullsampled" Whether to load the full or sub-sampled one. Please read notes section. - version : str, default "1.5.8.0" + version : str, default "1.5.9.3" Version of the OTBiolab+ software used (4 points). Tested versions are: "1.5.3.0", @@ -738,9 +774,15 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", If your specific version is not available in the tested versions, trying with the closer one usually works, but please double check the results. + extras : None or str, default None + Extras is used to store additional custom values. These information + will be stored in a pd.DataFrame with columns named as in the .mat + file. If not None, pass a regex pattern unequivocally identifying the + variable in the .mat file to load as extras. Returns ------- @@ -752,23 +794,29 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): - emg_from_otb : import the .mat file exportable by OTBiolab+. - emg_from_demuse : import the .mat file used in DEMUSE. - emg_from_customcsv : Import custom data from a .csv file. + - refsig_from_customcsv : Import the reference signal from a custom .csv. Notes --------- The returned file is called ``emg_refsig`` for convention. - The input .mat file exported from the OTBiolab+ software should contain: - - refsig signal: there should be the fullsampled or the subsampled + The input .mat file exported from the OTBiolab+ software must contain: + + - Reference signal: there must be the fullsampled or the subsampled version (in OTBioLab+ the "performed path" refers to the subsampled signal, the "acquired data" to the fullsampled signal), REF_SIGNAL is expected to be expressed as % of the MVC (but not compulsory). + - NO OTHER ELEMENTS SHOULD BE PRESENT, unless an appropriate regex pattern + is passed to 'extras='! Structure of the returned emg_refsig: + emg_refsig = { "SOURCE": SOURCE, "FILENAME": FILENAME, "FSAMP": FSAMP, "REF_SIGNAL": REF_SIGNAL, + "EXTRAS": EXTRAS, } Examples @@ -783,7 +831,7 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): mat_file = loadmat(filepath, simplify_cells=True) - # Parse .mat obtained from DEMUSE to see the available variables + # Parse .mat obtained from OTBiolab+ to see the available variables """ print( "\n--------------------------------\nAvailable dict keys are:\n\n{}\n".format( mat_file.keys() @@ -799,10 +847,11 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", ] if version not in valid_versions: raise ValueError( - f"Specified version is not valid. Use one of:\n{valid_versions}" + f"\nSpecified version is not valid. Use one of:\n{valid_versions}\n" ) if version in [ @@ -813,6 +862,7 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", ]: # Simplify (rename) columns description and extract all the parameters # in a pd.DataFrame @@ -823,28 +873,32 @@ def refsig_from_otb(filepath, refsig="fullsampled", version="1.5.8.0"): df = pd.DataFrame(mat_file["Data"], columns=col) + # Use this to know the data source and name of the file + SOURCE = "OTB_REFSIG" + FILENAME = os.path.basename(filepath) + FSAMP = float(mat_file["SamplingFrequency"]) + # Convert the input passed to refsig in a list compatible with the # function get_otb_refsignal refsig_ = [True, refsig] REF_SIGNAL = get_otb_refsignal(df=df, refsig=refsig_) - # Use this to know the data source and name of the file - SOURCE = "OTB_REFSIG" - FILENAME = os.path.basename(filepath) - FSAMP = int(mat_file["SamplingFrequency"]) + # Get EXTRAS + EXTRAS = get_otb_extras(df=df, extras=extras) - emg_refsig = { - "SOURCE": SOURCE, - "FILENAME": FILENAME, - "FSAMP": FSAMP, - "REF_SIGNAL": REF_SIGNAL, - } + emg_refsig = { + "SOURCE": SOURCE, + "FILENAME": FILENAME, + "FSAMP": FSAMP, + "REF_SIGNAL": REF_SIGNAL, + "EXTRAS": EXTRAS, + } - return emg_refsig + return emg_refsig # --------------------------------------------------------------------- -# Functions to open custom CSV documents. +# Function to load custom CSV documents. def emg_from_customcsv( filepath, ref_signal="REF_SIGNAL", @@ -852,11 +906,13 @@ def emg_from_customcsv( ipts="IPTS", mupulses="MUPULSES", binary_mus_firing="BINARY_MUS_FIRING", + accuracy="ACCURACY", + extras="EXTRAS", fsamp=2048, ied=8, ): """ - Import custom data from a .csv file. + Import the emgfile from a custom .csv file. The variables of interest should be contained in columns. The name of the columns containing each variable can be specified by the user if different @@ -869,7 +925,11 @@ def emg_from_customcsv( 'RAW_SIGNAL_2', ... , 'RAW_SIGNAL_n', the label of the columns should be 'RAW_SIGNAL'. If the parameters in input are not present in the .csv file, the user - can simply leave the original inputs. + should leave the original inputs. + + The .csv file must contain at least the raw_signal and one of 'mupulses' or + 'binary_mus_firing'. If 'mupulses' is absent, it will be calculated from + 'binary_mus_firing' and viceversa. Parameters ---------- @@ -878,19 +938,25 @@ def emg_from_customcsv( (including file extension .mat). This can be a simple string, the use of Path is not necessary. ref_signal : str, default 'REF_SIGNAL' - Label of the column(s) containing the reference signal. + Label of the column containing the reference signal. raw_signal : str, default 'RAW_SIGNAL' Label of the column(s) containing the raw emg signal. ipts : str, default 'IPTS' - Label of the column(s) containing the pulse train. + Label of the column(s) containing the pulse train (decomposed source). mupulses : str, default 'MUPULSES' Label of the column(s) containing the times of firing. binary_mus_firing : str, default 'BINARY_MUS_FIRING' Label of the column(s) containing the binary representation of the MUs firings. - fsamp : int, default 2048 + accuracy : str, default 'ACCURACY' + Label of the column(s) containing the accuracy score of the MUs + firings. + extras : str, default 'EXTRAS' + Label of the column(s) containing custom values. This information will + be stored in a pd.DataFrame with columns named as in the .csv file. + fsamp : int or float, default 2048 Tha sampling frequency. - ied : int, default 8 + ied : int or float, default 8 The inter-electrode distance in mm. Returns @@ -902,21 +968,22 @@ def emg_from_customcsv( -------- - emg_from_demuse : import the .mat file used in DEMUSE. - emg_from_otb : import the .mat file exportable by OTBiolab+. - - refsig_from_otb : import REF_SIGNAL in the .mat file exportable by + - refsig_from_otb : import reference signal in the .mat file exportable by OTBiolab+. + - refsig_from_customcsv : Import the reference signal from a custom .csv. Notes ----- The returned file is called ``emgfile`` for convention. Structure of the emgfile: + emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL + "ACCURACY": ACCURACY, "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -924,19 +991,20 @@ def emg_from_customcsv( "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, } Examples -------- An example of the .csv file to load: >>> - REF_SIGNAL RAW_SIGNAL (1) RAW_SIGNAL (2) RAW_SIGNAL (3) ... IPTS (1) IPTS (2) MUPULSES (1) MUPULSES (2) BINARY_MUS_FIRING (1) BINARY_MUS_FIRING (2) - 0 1 0.100000 0.100000 0.100000 ... 0.010000 0.010000 2.0 1.0 0 0 - 1 2 2.000000 2.000000 2.000000 ... 0.001000 0.001000 5.0 2.0 0 0 - 2 3 0.500000 0.500000 0.500000 ... 0.020000 0.020000 8.0 9.0 0 0 - 3 4 0.150000 0.150000 0.150000 ... 0.002000 0.002000 9.0 15.0 0 1 - 4 5 0.350000 0.350000 0.350000 ... -0.100000 -0.100000 15.0 18.0 1 1 - 5 6 0.215000 0.215000 0.215000 ... 0.200000 0.200000 16.0 NaN 1 0 + REF_SIGNAL RAW_SIGNAL (1) RAW_SIGNAL (2) RAW_SIGNAL (3) RAW_SIGNAL (4) ... MUPULSES (2) BINARY_MUS_FIRING (1) BINARY_MUS_FIRING (2) ACCURACY (1) ACCURACY (2) + 1 0.100000 0.100000 0.100000 0.100000 ... 1.0 0 0 0.89 0.95 + 2 2.000000 2.000000 2.000000 2.000000 ... 2.0 0 0 + 3 0.500000 0.500000 0.500000 0.500000 ... 9.0 0 0 + 4 0.150000 0.150000 0.150000 0.150000 ... 15.0 0 1 + 5 0.350000 0.350000 0.350000 0.350000 ... 18.0 1 1 + 6 0.215000 0.215000 0.215000 0.215000 ... 22.0 1 0 For an extended explanation of the imported emgfile use: @@ -949,99 +1017,235 @@ def emg_from_customcsv( # Load the csv csv = pd.read_csv(filepath) - # Get REF_SIGNAL - REF_SIGNAL = csv.filter(regex=ref_signal, axis=1) - if not REF_SIGNAL.empty: - REF_SIGNAL.columns = [i for i in range(len(REF_SIGNAL.columns))] - else: - warnings.warn( - "\nref_signal not found, it might be necessary for some analysis\n" - ) - REF_SIGNAL = np.nan + # First, get the basic information and compulsory variables (i.e., + # RAW_SIGNAL, MUPULSES, BINARY_MUS_FIRING). + + # Use this to know the data source and name of the file + SOURCE = "CUSTOMCSV" + FILENAME = os.path.basename(filepath) # Get RAW_SIGNAL - RAW_SIGNAL = csv.filter(regex=raw_signal, axis=1) + RAW_SIGNAL = csv.filter(regex=raw_signal, axis=1).dropna() if not RAW_SIGNAL.empty: RAW_SIGNAL.columns = [i for i in range(len(RAW_SIGNAL.columns))] else: - warnings.warn( - "\nraw_signal not found, it might be necessary for some analysis\n" + raise ValueError( + "\nraw_signal not found\n" ) - RAW_SIGNAL = np.nan - # Get IPTS - IPTS = csv.filter(regex=ipts, axis=1) - if not IPTS.empty: - IPTS.columns = [i for i in range(len(IPTS.columns))] - else: - warnings.warn( - "\nipts not found, it might be necessary for some analysis\n" - ) - IPTS = np.nan + # Get MUPULSES/BINARY_MUS_FIRING + df_mupulses = csv.filter(regex=mupulses, axis=1) + BINARY_MUS_FIRING = csv.filter(regex=binary_mus_firing, axis=1).dropna() - # Get MUPULSES - df = csv.filter(regex=mupulses, axis=1) - if not df.empty: + if df_mupulses.empty and BINARY_MUS_FIRING.empty: + raise ValueError( + "\nmupulses and binary_mus_firing not found. At least one of the two must be present\n") + elif not df_mupulses.empty and not BINARY_MUS_FIRING.empty: MUPULSES = [] - for col in df.columns: - toappend = df[col].dropna().to_numpy(dtype=int) + for col in df_mupulses.columns: + toappend = df_mupulses[col].dropna().to_numpy(dtype=int) MUPULSES.append(toappend) - else: - MUPULSES = np.nan - # Get BINARY_MUS_FIRING - BINARY_MUS_FIRING = csv.filter(regex=binary_mus_firing, axis=1) - if not BINARY_MUS_FIRING.empty: BINARY_MUS_FIRING.columns = [ i for i in range(len(BINARY_MUS_FIRING.columns)) ] - else: - BINARY_MUS_FIRING = np.nan - # Get EMG_LENGTH and NUMBER_OF_MUS - EMG_LENGTH, NUMBER_OF_MUS = IPTS.shape + elif df_mupulses.empty and not BINARY_MUS_FIRING.empty: + BINARY_MUS_FIRING.columns = [ + i for i in range(len(BINARY_MUS_FIRING.columns)) + ] - # Use this to know the data source and name of the file - SOURCE = "CUSTOM" - FILENAME = os.path.basename(filepath) + MUPULSES = mupulses_from_binary(binarymusfiring=BINARY_MUS_FIRING) - if NUMBER_OF_MUS > 0: - # Calculate the PNR - to_append = [] - for mu in range(NUMBER_OF_MUS): - pnr = compute_pnr( - ipts=IPTS[mu], - mupulses=MUPULSES[mu], - fsamp=fsamp, - ) - to_append.append(pnr) - PNR = pd.DataFrame(to_append) + else: # if not df_mupulses.empty and BINARY_MUS_FIRING.empty: + MUPULSES = [] + for col in df_mupulses.columns: + toappend = df_mupulses[col].dropna().to_numpy(dtype=int) + MUPULSES.append(toappend) - # Calculate the SIL - to_append = [] - for mu in range(NUMBER_OF_MUS): - sil = compute_sil(ipts=IPTS[mu], mupulses=MUPULSES[mu]) - to_append.append(sil) - SIL = pd.DataFrame(to_append) + l, _ = RAW_SIGNAL.shape + BINARY_MUS_FIRING = create_binary_firings( + emg_length=l, + number_of_mus=len(MUPULSES), + mupulses=MUPULSES, + ) + + # Get EMG_LENGTH and NUMBER_OF_MUS + EMG_LENGTH, NUMBER_OF_MUS = BINARY_MUS_FIRING.shape + + # Second, get/generate the other variables + # Get REF_SIGNAL + REF_SIGNAL = csv.filter(regex=ref_signal, axis=1).dropna() + if not REF_SIGNAL.empty: + REF_SIGNAL.columns = [i for i in range(len(REF_SIGNAL.columns))] + if len(REF_SIGNAL.columns) > 1: + warnings.warn( + "\nMore than 1 reference signal detected. You should place other signals in 'EXTRAS'\n" + ) + else: + REF_SIGNAL = pd.DataFrame(columns=[0]) + warnings.warn( + "\nref_signal not found, it might be necessary for some analyses\n" + ) # returns empty pd.DataFrame with 1 column + # Get IPTS + IPTS = csv.filter(regex=ipts, axis=1).dropna() + if not IPTS.empty: + IPTS.columns = [i for i in range(len(IPTS.columns))] + else: + IPTS = pd.DataFrame(columns=[*range(NUMBER_OF_MUS)]) + warnings.warn( + "\nipts not found, it might be necessary for some analyses\n" + ) # returns empty pd.DataFrame with n columns + + # Get ACCURACY + ACCURACY = csv.filter(regex=accuracy, axis=1).dropna() + if not ACCURACY.empty: + # Merge all the accuracies of each MU in a single column. + ACCURACY = ACCURACY.melt(value_name=0).drop(labels="variable", axis=1) else: - PNR = np.nan - SIL = np.nan + ACCURACY = pd.DataFrame(columns=[0]) + warnings.warn( + "\naccuracy not found. It might be necessary for some analyses\n" + ) # returns empty pd.DataFrame with 1 column + + # Get EXTRAS + EXTRAS = csv.filter(regex=extras, axis=1) + if EXTRAS.empty: + EXTRAS = pd.DataFrame(columns=[0]) + # returns empty pd.DataFrame with 1 column emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL, + "ACCURACY": ACCURACY, "IPTS": IPTS, "MUPULSES": MUPULSES, - "FSAMP": fsamp, - "IED": ied, + "FSAMP": float(fsamp), + "IED": float(ied), "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, + } + + return emgfile + + +# --------------------------------------------------------------------- +# Function to load the reference signal from custom CSV documents. + +def refsig_from_customcsv( + filepath, + ref_signal="REF_SIGNAL", + extras="EXTRAS", + fsamp=2048, +): + """ + Import the reference signal from a custom .csv file. + + Compared to the function emg_from_customcsv, this function only imports the + REF_SIGNAL and, therefore, it can be used for special cases where only the + REF_SIGNAL is necessary. This will allow for a faster execution of the + script and to avoid exceptions for missing data. + + This function detects the content of the .csv by parsing the .csv columns. + For parsing, column labels should be provided. A label is a term common + to all the columns containing the same information. + For example, if the ref signal is contained in the column 'REF_SIGNAL', the + label of the columns should be 'REF_SIGNAL' or a part of it (e.g., 'REF'). + If the parameters in input are not present in the .csv file (e.g., + 'EXTRAS'), the user should leave the original inputs. + + Parameters + ---------- + filepath : str or Path + The directory and the name of the file to load + (including file extension .mat). + This can be a simple string, the use of Path is not necessary. + ref_signal : str, default 'REF_SIGNAL' + Label of the column containing the reference signal. + extras : str, default 'EXTRAS' + Label of the column(s) containing custom values. These information + will be stored in a pd.DataFrame with columns named as in the .csv + file. + fsamp : int or float, default 2048 + Tha sampling frequency. + + Returns + ------- + emg_refsig : dict + A dictionary containing all the useful variables. + + Notes + --------- + The returned file is called ``emg_refsig`` for convention. + + Structure of the returned emg_refsig: + + emg_refsig = { + "SOURCE": SOURCE, + "FILENAME": FILENAME, + "FSAMP": FSAMP, + "REF_SIGNAL": REF_SIGNAL, + "EXTRAS": EXTRAS, + } + + Examples + -------- + An example of the .csv file to load: + >>> + REF_SIGNAL EXTRAS (1) EXTRAS (2) + 1 0.1 0 + 2 0.2 0 + 3 0.3 0 + 4 0.4 0 + 5 0.5 1 + 6 0.6 1 + + For an extended explanation of the imported emgfile use: + + >>> import openhdemg.library as emg + >>> emgfile = refsig_from_customcsv(filepath = "mypath/file.csv") + >>> info = emg.info() + >>> info.data(emgfile) + """ + + # Load the csv + csv = pd.read_csv(filepath) + + # Use this to know the data source and name of the file + SOURCE = "CUSTOMCSV_REFSIG" + FILENAME = os.path.basename(filepath) + + # Get REF_SIGNAL + REF_SIGNAL = csv.filter(regex=ref_signal, axis=1).dropna() + if not REF_SIGNAL.empty: + REF_SIGNAL.columns = [i for i in range(len(REF_SIGNAL.columns))] + if len(REF_SIGNAL.columns) > 1: + warnings.warn( + "\nMore than 1 reference signal detected. You should place other signals in 'EXTRAS'\n" + ) + else: + REF_SIGNAL = pd.DataFrame(columns=[0]) + warnings.warn( + "\nref_signal not found, it might be necessary for some analyses\n" + ) # returns empty pd.DataFrame with 1 column + + # Get EXTRAS + EXTRAS = csv.filter(regex=extras, axis=1) + if EXTRAS.empty: + EXTRAS = pd.DataFrame(columns=[0]) + # returns empty pd.DataFrame with 1 column + + emgfile = { + "SOURCE": SOURCE, + "FILENAME": FILENAME, + "FSAMP": float(fsamp), + "REF_SIGNAL": REF_SIGNAL, + "EXTRAS": EXTRAS, } return emgfile @@ -1064,7 +1268,7 @@ def save_json_emgfile(emgfile, filepath): This can be a simple string; The use of Path is not necessary. """ - if emgfile["SOURCE"] in ["DEMUSE", "OTB", "CUSTOM"]: + if emgfile["SOURCE"] in ["DEMUSE", "OTB", "CUSTOMCSV"]: """ We need to convert all the components of emgfile to a dictionary and then to json object. @@ -1076,8 +1280,7 @@ def save_json_emgfile(emgfile, filepath): "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL + "ACCURACY": ACCURACY "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -1085,6 +1288,7 @@ def save_json_emgfile(emgfile, filepath): "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, } """ # str or int @@ -1102,37 +1306,38 @@ def save_json_emgfile(emgfile, filepath): emg_length = json.dumps(emg_length) number_of_mus = json.dumps(number_of_mus) - # Extract the df from the dict, convert the dict to a json, put the + # df + # Extract the df from the dict, convert the df to a json, put the # json in a dict, convert the dict to a json. # We use dict converted to json to locate better the objects while # re-importing them in python. raw_signal = emgfile["RAW_SIGNAL"] ref_signal = emgfile["REF_SIGNAL"] - pnr = emgfile["PNR"] - sil = emgfile["SIL"] + accuracy = emgfile["ACCURACY"] ipts = emgfile["IPTS"] binary_mus_firing = emgfile["BINARY_MUS_FIRING"] + extras = emgfile["EXTRAS"] raw_signal = raw_signal.to_json() ref_signal = ref_signal.to_json() - pnr = pnr.to_json() - sil = sil.to_json() + accuracy = accuracy.to_json() ipts = ipts.to_json() binary_mus_firing = binary_mus_firing.to_json() + extras = extras.to_json() raw_signal = {"RAW_SIGNAL": raw_signal} ref_signal = {"REF_SIGNAL": ref_signal} - pnr = {"PNR": pnr} - sil = {"SIL": sil} + accuracy = {"ACCURACY": accuracy} ipts = {"IPTS": ipts} binary_mus_firing = {"BINARY_MUS_FIRING": binary_mus_firing} + extras = {"EXTRAS": extras} raw_signal = json.dumps(raw_signal) ref_signal = json.dumps(ref_signal) - pnr = json.dumps(pnr) - sil = json.dumps(sil) + accuracy = json.dumps(accuracy) ipts = json.dumps(ipts) binary_mus_firing = json.dumps(binary_mus_firing) + extras = json.dumps(extras) # list of ndarray. # Every array has to be converted in a list; then, the list of lists @@ -1151,8 +1356,7 @@ def save_json_emgfile(emgfile, filepath): filename, raw_signal, ref_signal, - pnr, - sil, + accuracy, ipts, mupulses, fsamp, @@ -1160,6 +1364,7 @@ def save_json_emgfile(emgfile, filepath): emg_length, number_of_mus, binary_mus_firing, + extras, ] json_to_save = json.dumps(list_to_save) @@ -1173,13 +1378,14 @@ def save_json_emgfile(emgfile, filepath): # To improve writing time, f.write is the bottleneck but it is # hard to improve. - elif emgfile["SOURCE"] == "OTB_REFSIG": + elif emgfile["SOURCE"] in ["OTB_REFSIG", "CUSTOMCSV_REFSIG"]: """ refsig = { "SOURCE" : SOURCE, "FILENAME": FILENAME, "FSAMP" : FSAMP, "REF_SIGNAL" : REF_SIGNAL, + "EXTRAS": EXTRAS, } """ # str or int @@ -1194,8 +1400,12 @@ def save_json_emgfile(emgfile, filepath): ref_signal = ref_signal.to_json() ref_signal = {"REF_SIGNAL": ref_signal} ref_signal = json.dumps(ref_signal) + extras = emgfile["EXTRAS"] + extras = extras.to_json() + extras = {"EXTRAS": extras} + extras = json.dumps(extras) # Merge all the objects in one - list_to_save = [source, filename, fsamp, ref_signal] + list_to_save = [source, filename, fsamp, ref_signal, extras] json_to_save = json.dumps(list_to_save) # Compress and save with gzip.open(filepath, "w") as f: @@ -1203,7 +1413,7 @@ def save_json_emgfile(emgfile, filepath): f.write(json_bytes) else: - raise Exception("File source not recognised") + raise ValueError("\nFile source not recognised\n") def emg_from_json(filepath): @@ -1233,14 +1443,14 @@ def emg_from_json(filepath): Notes ----- The returned file is called ``emgfile`` for convention - (or ``emg_refsig`` if SOURCE = "OTB_REFSIG"). + (or ``emg_refsig`` if SOURCE in ["OTB_REFSIG", "CUSTOMCSV_REFSIG"]). Examples -------- For an extended explanation of the imported emgfile use: >>> import openhdemg.library as emg - >>> emgfile = emg.askopenfile() + >>> emgfile = emg.emg_from_json(filepath="path/filename.json") >>> info = emg.info() >>> info.data(emgfile) """ @@ -1256,7 +1466,7 @@ def emg_from_json(filepath): print(type(jsonemgfile)) print(len(jsonemgfile)) - 11 + 13 """ # Access the dictionaries and extract the data # jsonemgfile[0] contains the SOURCE in a dictionary @@ -1266,7 +1476,7 @@ def emg_from_json(filepath): filename_dict = json.loads(jsonemgfile[1]) filename = filename_dict["FILENAME"] - if source in ["DEMUSE", "OTB", "CUSTOM"]: + if source in ["DEMUSE", "OTB", "CUSTOMCSV"]: # jsonemgfile[2] contains the RAW_SIGNAL in a dictionary, it can be # extracted in a new dictionary and converted into a pd.DataFrame. # index and columns are imported as str, we need to convert it to int. @@ -1283,60 +1493,64 @@ def emg_from_json(filepath): ref_signal.columns = ref_signal.columns.astype(int) ref_signal.index = ref_signal.index.astype(int) ref_signal.sort_index(inplace=True) - # jsonemgfile[4] contains the PNR to be treated as jsonemgfile[2] - pnr_dict = json.loads(jsonemgfile[4]) - pnr_dict = json.loads(pnr_dict["PNR"]) - pnr = pd.DataFrame(pnr_dict) - pnr.columns = pnr.columns.astype(int) - pnr.index = pnr.index.astype(int) - pnr.sort_index(inplace=True) - # jsonemgfile[5] contains the SIL to be treated as jsonemgfile[2] - sil_dict = json.loads(jsonemgfile[5]) - sil_dict = json.loads(sil_dict["SIL"]) - sil = pd.DataFrame(sil_dict) - sil.columns = sil.columns.astype(int) - sil.index = sil.index.astype(int) - sil.sort_index(inplace=True) - # jsonemgfile[6] contains the IPTS to be treated as jsonemgfile[2] - ipts_dict = json.loads(jsonemgfile[6]) + # jsonemgfile[4] contains the ACCURACY to be treated as jsonemgfile[2] + accuracy_dict = json.loads(jsonemgfile[4]) + accuracy_dict = json.loads(accuracy_dict["ACCURACY"]) + accuracy = pd.DataFrame(accuracy_dict) + accuracy.columns = accuracy.columns.astype(int) + accuracy.index = accuracy.index.astype(int) + accuracy.sort_index(inplace=True) + # jsonemgfile[5] contains the IPTS to be treated as jsonemgfile[2] + ipts_dict = json.loads(jsonemgfile[5]) ipts_dict = json.loads(ipts_dict["IPTS"]) ipts = pd.DataFrame(ipts_dict) ipts.columns = ipts.columns.astype(int) ipts.index = ipts.index.astype(int) ipts.sort_index(inplace=True) - # jsonemgfile[7] contains the MUPULSES which is a list of lists but + # jsonemgfile[6] contains the MUPULSES which is a list of lists but # has to be converted in a list of ndarrays. - mupulses = json.loads(jsonemgfile[7]) + mupulses = json.loads(jsonemgfile[6]) for num, element in enumerate(mupulses): mupulses[num] = np.array(element) - # jsonemgfile[8] contains the FSAMP to be treated as jsonemgfile[0] - fsamp_dict = json.loads(jsonemgfile[8]) - fsamp = int(fsamp_dict["FSAMP"]) - # jsonemgfile[9] contains the IED to be treated as jsonemgfile[0] - ied_dict = json.loads(jsonemgfile[9]) - ied = int(ied_dict["IED"]) - # jsonemgfile[10] contains the EMG_LENGTH to be treated as jsonemgfile[0] - emg_length_dict = json.loads(jsonemgfile[10]) + # jsonemgfile[7] contains the FSAMP to be treated as jsonemgfile[0] + fsamp_dict = json.loads(jsonemgfile[7]) + fsamp = float(fsamp_dict["FSAMP"]) + # jsonemgfile[8] contains the IED to be treated as jsonemgfile[0] + ied_dict = json.loads(jsonemgfile[8]) + ied = float(ied_dict["IED"]) + # jsonemgfile[9] contains the EMG_LENGTH to be treated as + # jsonemgfile[0] + emg_length_dict = json.loads(jsonemgfile[9]) emg_length = int(emg_length_dict["EMG_LENGTH"]) - # jsonemgfile[11] contains the NUMBER_OF_MUS to be treated as + # jsonemgfile[10] contains the NUMBER_OF_MUS to be treated as # jsonemgfile[0] - number_of_mus_dict = json.loads(jsonemgfile[11]) + number_of_mus_dict = json.loads(jsonemgfile[10]) number_of_mus = int(number_of_mus_dict["NUMBER_OF_MUS"]) - # jsonemgfile[12] contains the BINARY_MUS_FIRING to be treated as + # jsonemgfile[11] contains the BINARY_MUS_FIRING to be treated as # jsonemgfile[2] - binary_mus_firing_dict = json.loads(jsonemgfile[12]) - binary_mus_firing_dict = json.loads(binary_mus_firing_dict["BINARY_MUS_FIRING"]) + binary_mus_firing_dict = json.loads(jsonemgfile[11]) + binary_mus_firing_dict = json.loads( + binary_mus_firing_dict["BINARY_MUS_FIRING"] + ) binary_mus_firing = pd.DataFrame(binary_mus_firing_dict) binary_mus_firing.columns = binary_mus_firing.columns.astype(int) binary_mus_firing.index = binary_mus_firing.index.astype(int) + # jsonemgfile[12] contains the EXTRAS to be treated as + # jsonemgfile[2] + extras_dict = json.loads(jsonemgfile[12]) + extras_dict = json.loads(extras_dict["EXTRAS"]) + extras = pd.DataFrame(extras_dict) + # extras.columns = extras.columns.astype(int) + # extras.index = extras.index.astype(int) + # extras.sort_index(inplace=True) + # Don't alter extras, leave that to the user for maximum control emgfile = { "SOURCE": source, "FILENAME": filename, "RAW_SIGNAL": raw_signal, "REF_SIGNAL": ref_signal, - "PNR": pnr, - "SIL": sil, + "ACCURACY": accuracy, "IPTS": ipts, "MUPULSES": mupulses, "FSAMP": fsamp, @@ -1344,12 +1558,13 @@ def emg_from_json(filepath): "EMG_LENGTH": emg_length, "NUMBER_OF_MUS": number_of_mus, "BINARY_MUS_FIRING": binary_mus_firing, + "EXTRAS": extras, } - elif source == "OTB_REFSIG": + elif source in ["OTB_REFSIG", "CUSTOMCSV_REFSIG"]: # jsonemgfile[2] contains the fsamp fsamp_dict = json.loads(jsonemgfile[2]) - fsamp = int(fsamp_dict["FSAMP"]) + fsamp = float(fsamp_dict["FSAMP"]) # jsonemgfile[3] contains the REF_SIGNAL ref_signal_dict = json.loads(jsonemgfile[3]) ref_signal_dict = json.loads(ref_signal_dict["REF_SIGNAL"]) @@ -1357,16 +1572,21 @@ def emg_from_json(filepath): ref_signal.columns = ref_signal.columns.astype(int) ref_signal.index = ref_signal.index.astype(int) ref_signal.sort_index(inplace=True) + # jsonemgfile[4] contains the EXTRAS + extras_dict = json.loads(jsonemgfile[4]) + extras_dict = json.loads(extras_dict["EXTRAS"]) + extras = pd.DataFrame(extras_dict) emgfile = { "SOURCE": source, "FILENAME": filename, "FSAMP": fsamp, "REF_SIGNAL": ref_signal, + "EXTRAS": extras, } else: - raise Exception("File source not recognised") + raise Exception("\nFile source not recognised\n") return emgfile @@ -1383,9 +1603,12 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): initialdir : str or Path, default "/" The directory of the file to load (excluding file name). This can be a simple string, the use of Path is not necessary. - filesource : str {"OPENHDEMG", "DEMUSE", "OTB", "OTB_REFSIG", "CUSTOM"}, default "OPENHDEMG" - See notes for how files should be exported from OTB. + filesource : str {"OPENHDEMG", "DEMUSE", "OTB", "OTB_REFSIG", "CUSTOMCSV", CUSTOMCSV_REFSIG}, default "OPENHDEMG" + The source of the file. See notes for how files should be exported + from OTB. + ``OPENHDEMG`` + File saved from openhdemg (.json). ``DEMUSE`` File saved from DEMUSE (.mat). ``OTB`` @@ -1393,10 +1616,10 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): (.mat). ``OTB_REFSIG`` File exported from OTB with only the reference signal (.mat). - ``CUSTOM`` + ``CUSTOMCSV`` Custom file format (.csv). - ``OPENHDEMG`` - File saved from openhdemg (.json). + ``CUSTOMCSV_REFSIG`` + Custom file format (.csv) containing only the reference signal. otb_ext_factor : int, default 8 The extension factor used for the decomposition in the OTbiolab+ software. @@ -1406,7 +1629,7 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): or sub-sampled one. The list is composed as [bool, str]. str can be "fullsampled" or "subsampled". Ignore if loading other files. - otb_version : str, default "1.5.8.0" + otb_version : str, default "1.5.9.3" Version of the OTBiolab+ software used (4 points). Tested versions are: "1.5.3.0", @@ -1416,6 +1639,7 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): "1.5.7.2", "1.5.7.3", "1.5.8.0", + "1.5.9.3", If your specific version is not available in the tested versions, trying with the closer one usually works, but please double check the results. Ignore if loading other files. @@ -1437,6 +1661,15 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): Label of the column(s) containing the binary representation of the MUs firings of the custom file. Ignore if loading other files. + custom_accuracy : str, default 'ACCURACY' + Label of the column(s) containing the accuracy score of the + decomposed MUs in the custom file. + Ignore if loading other files. + custom_extras : str, default 'EXTRAS' + Label of the column(s) containing custom values in the custom file. + This information will be stored in a pd.DataFrame with columns named + as in the .csv file. + Ignore if loading other files. custom_fsamp : int, default 2048 Tha sampling frequency of the custom file. Ignore if loading other files. @@ -1456,10 +1689,11 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): Notes ----- The returned file is called ``emgfile`` for convention (or ``emg_refsig`` - if SOURCE = "OTB_REFSIG"). + if SOURCE in ["OTB_REFSIG", CUSTOMCSV_REFSIG]). The input .mat file exported from the OTBiolab+ software should have a specific content: + - refsig signal is optional but, if present, there should be both the fullsampled and the subsampled version (in OTBioLab+ the "performed path" refers to the subsampled signal, the "acquired data" to the @@ -1471,7 +1705,8 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): - The raw EMG signal should be present (it has no specific name in OTBioLab+) with all the channels. Don't exclude unwanted channels before exporting the .mat file. - - NO OTHER ELEMENTS SHOULD BE PRESENT! + - NO OTHER ELEMENTS SHOULD BE PRESENT! unless an appropriate regex pattern + is passed to 'extras='! For custom .csv files: The variables of interest should be contained in columns. The name of the @@ -1487,15 +1722,17 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): can simply leave the original inputs. Please see the documentation of the function emg_from_customcsv for additional informations. + The .csv file must contain all the variables. The only admitted exceptions + are 'ref_signal' and 'ipts'. Structure of the returned emgfile: + emgfile = { "SOURCE": SOURCE, "FILENAME": FILENAME, "RAW_SIGNAL": RAW_SIGNAL, "REF_SIGNAL": REF_SIGNAL, - "PNR": PNR, - "SIL": SIL, + "ACCURACY": accuracy score (depending on source file type), "IPTS": IPTS, "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -1503,14 +1740,17 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): "EMG_LENGTH": EMG_LENGTH, "NUMBER_OF_MUS": NUMBER_OF_MUS, "BINARY_MUS_FIRING": BINARY_MUS_FIRING, + "EXTRAS": EXTRAS, } Structure of the returned emg_refsig: + emg_refsig = { "SOURCE": SOURCE, "FILENAME": FILENAME, "FSAMP": FSAMP, "REF_SIGNAL": REF_SIGNAL, + "EXTRAS": EXTRAS, } Examples @@ -1518,12 +1758,13 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): For an extended explanation of the imported emgfile use: >>> import openhdemg.library as emg - >>> emgfile = emg.askopenfile() + >>> emgfile = emg.askopenfile(filesource="OPENHDEMG") >>> info = emg.info() >>> info.data(emgfile) """ - # Set initialdir (actually not working on Windows) + # Set initialdir (actually not working on Windows, but it's not a problem + # of the code implementation) if isinstance(initialdir, str): if initialdir == "/": initialdir = "/Decomposed Test files/" @@ -1537,23 +1778,23 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): file_toOpen = filedialog.askopenfilename( initialdir=initialdir, title=f"Select a {filesource} file to load", - filetypes=[("MATLAB files", ".mat")], + filetypes=[("MATLAB files", "*.mat")], ) elif filesource == "OPENHDEMG": file_toOpen = filedialog.askopenfilename( initialdir=initialdir, title="Select an OPENHDEMG file to load", - filetypes=[("JSON files", ".json")], + filetypes=[("JSON files", "*.json")], ) - elif filesource == "CUSTOM": # TODO add custom_refignal + elif filesource in ["CUSTOMCSV", "CUSTOMCSV_REFSIG"]: file_toOpen = filedialog.askopenfilename( initialdir=initialdir, title="Select a custom file to load", - filetypes=[("CSV files", ".csv")], + filetypes=[("CSV files", "*.csv")], ) else: raise Exception( - "filesource not valid, it must be one of 'DEMUSE', 'OTB', 'OTB_REFSIG' or 'OPENHDEMG'" + "\nfilesource not valid, it must be one of 'DEMUSE', 'OTB', 'OTB_REFSIG', 'OPENHDEMG', 'CUSTOMCSV', 'CUSTOMCSV_REFSIG'\n" ) # Destroy the root since it is no longer necessary @@ -1567,18 +1808,18 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): filepath=file_toOpen, ext_factor=kwargs.get("otb_ext_factor", 8), refsig=kwargs.get("otb_refsig_type", [True, "fullsampled"]), - version=kwargs.get("otb_version", "1.5.8.0") + version=kwargs.get("otb_version", "1.5.9.3") ) elif filesource == "OTB_REFSIG": ref = kwargs.get("otb_refsig_type", [True, "fullsampled"]) emgfile = refsig_from_otb( filepath=file_toOpen, refsig=ref[1], - version=kwargs.get("otb_version", "1.5.8.0"), + version=kwargs.get("otb_version", "1.5.9.3"), ) elif filesource == "OPENHDEMG": emgfile = emg_from_json(filepath=file_toOpen) - else: # custom + elif filesource == "CUSTOMCSV": emgfile = emg_from_customcsv( filepath=file_toOpen, ref_signal=kwargs.get("custom_ref_signal", "REF_SIGNAL"), @@ -1589,9 +1830,18 @@ def askopenfile(initialdir="/", filesource="OPENHDEMG", **kwargs): "custom_binary_mus_firing", "BINARY_MUS_FIRING" ), + accuracy=kwargs.get("custom_accuracy", "ACCURACY"), + extras=kwargs.get("custom_extras", "EXTRAS"), fsamp=kwargs.get("custom_fsamp", 2048), ied=kwargs.get("custom_ied", 8), ) + else: # CUSTOMCSV_REFSIG + emgfile = refsig_from_customcsv( + filepath=file_toOpen, + ref_signal=kwargs.get("custom_ref_signal", "REF_SIGNAL"), + extras=kwargs.get("custom_extras", "EXTRAS"), + fsamp=kwargs.get("custom_fsamp", 2048), + ) print("\n-----------\nFile loaded\n-----------\n") @@ -1619,7 +1869,7 @@ def asksavefile(emgfile): filepath = filedialog.asksaveasfilename( defaultextension=".json", - filetypes=[("json files", "*.json")], + filetypes=[("JSON files", "*.json")], title="Save JSON file", ) diff --git a/openhdemg/library/plotemg.py b/openhdemg/library/plotemg.py index eb63e3a..2f880ad 100644 --- a/openhdemg/library/plotemg.py +++ b/openhdemg/library/plotemg.py @@ -1,6 +1,6 @@ """ -This module contains all the functions used to visualise the emg file, -the MUs properties or to save figures. +This module contains all the functions used to visualise the content of the +imported EMG file, the MUs properties or to save figures. """ import matplotlib.pyplot as plt @@ -58,7 +58,7 @@ def plot_emgsig( """ Plot the RAW_SIGNAL. Single or multiple channels. - Up to 12 channels (a common matrix row) can be easily observed togheter + Up to 12 channels (a common matrix row) can be easily observed togheter, but more can be plotted. Parameters @@ -423,10 +423,10 @@ def plot_mupulses( ---------- emgfile : dict The dictionary containing the emgfile. - munumber : str {"all"}, int or list, default "all" + munumber : str, int or list, default "all" + ``all`` IPTS of all the MUs is plotted. - Otherwise, a single MU (int) or multiple MUs (list of int) can be specified. The list can be passed as a manually-written list or with: @@ -590,10 +590,9 @@ def plot_ipts( ---------- emgfile : dict The dictionary containing the emgfile. - munumber : str {"all"}, int or list, default "all" + munumber : str, int or list, default "all" ``all`` IPTS of all the MUs is plotted. - Otherwise, a single MU (int) or multiple MUs (list of int) can be specified. The list can be passed as a manually-written list or with: @@ -877,7 +876,7 @@ def plot_idr( ax1.set_ylabel("Motor units") ax1.set_xlabel("Time (Sec)" if timeinseconds else "Samples") - else: + elif len(munumber) == 1: ax1 = sns.scatterplot( x=idr[munumber[0]]["timesec" if timeinseconds else "mupulses"], y=idr[munumber[0]]["idr"], @@ -950,6 +949,7 @@ def plot_muaps( ----- There is no limit to the number of MUs and STA files that can be overplotted. + ``Remember: the different STAs should be matched`` with same number of electrode, processing (i.e., differential) and computed on the same timewindow. @@ -1131,7 +1131,7 @@ def plot_muap( -------- Plot all the consecutive MUAPs of a single MU. In this case we are plotting the matrix channel 45 which is placed in - column 4 ("col3") as Python numbering is base 0. + column 4 ("col3" as Python numbering is base 0). >>> import openhdemg.library as emg >>> emgfile = emg.askopenfile(filesource="OTB", otb_ext_factor=8) diff --git a/openhdemg/library/tools.py b/openhdemg/library/tools.py index eb1cac7..95bd094 100644 --- a/openhdemg/library/tools.py +++ b/openhdemg/library/tools.py @@ -11,7 +11,7 @@ import matplotlib.pyplot as plt from scipy import signal import warnings -from openhdemg.library.mathtools import compute_pnr, compute_sil +from openhdemg.library.mathtools import compute_sil def showselect(emgfile, title="", titlesize=12, nclic=2): @@ -99,8 +99,7 @@ def create_binary_firings(emg_length, number_of_mus, mupulses): Returns ------- binary_MUs_firing : pd.DataFrame - A pd.DataFrame containing the binary representation of MUs firing or - np.nan if the variable was not found. + A pd.DataFrame containing the binary representation of MUs firing. """ # skip the step if I don't have the mupulses (is nan) @@ -123,10 +122,43 @@ def create_binary_firings(emg_length, number_of_mus, mupulses): return binary_MUs_firing else: - return np.nan + raise ValueError("mupulses is not a list of ndarrays") -def resize_emgfile(emgfile, area=None): +def mupulses_from_binary(binarymusfiring): + """ + Extract the MUPULSES from the binary MUs firings. + + Parameters + ---------- + binarymusfiring : pd.DataFrame + A pd.DataFrame containing the binary representation of MUs firings. + + Returns + ------- + MUPULSES : list + A list of ndarrays containing the firing time of each MU. + """ + + # Create empty list of lists to fill with ndarrays containing the MUPULSES + # (point of firing) + numberofMUs = len(binarymusfiring.columns) + MUPULSES = [[] for _ in range(numberofMUs)] + + for i in binarymusfiring: # Loop all the MUs + my_ndarray = [] + for idx, x in binarymusfiring[i].items(): # Loop the MU firing times + if x > 0: + my_ndarray.append(idx) + # Take the firing time and add it to the ndarray + + my_ndarray = np.array(my_ndarray) + MUPULSES[i] = my_ndarray + + return MUPULSES + + +def resize_emgfile(emgfile, area=None, accuracy="recalculate"): """ Resize all the emgfile. @@ -141,6 +173,13 @@ def resize_emgfile(emgfile, area=None): The resizing area. If already known, it can be passed in samples, as a list (e.g., [120,2560]). If None, the user can select the area of interest manually. + accuracy : str {"recalculate", "maintain"}, default "recalculate" + ``recalculate`` + The Silhouette score is computed in the new resized file. This can + be done only if IPTS is present. + ``maintain`` + The original accuracy measure already contained in the emgfile is + returned without any computation. Returns ------- @@ -152,8 +191,6 @@ def resize_emgfile(emgfile, area=None): Notes ----- Suggested names for the returned objects: rs_emgfile, start_, end_. - - PNR and SIL are computed again in the new resized area. """ # Identify the area of interest @@ -173,14 +210,13 @@ def resize_emgfile(emgfile, area=None): # Create the object to store the resized emgfile. rs_emgfile = copy.deepcopy(emgfile) """ - PNR and SIL should be re-computed on the new portion of the file. + ACCURACY should be re-computed on the new portion of the file if possible. Need to be resized: ==> emgfile = { "SOURCE": SOURCE, ==> "RAW_SIGNAL": RAW_SIGNAL, ==> "REF_SIGNAL": REF_SIGNAL, - ==> "PNR": PNR, - ==> "SIL": SIL, + ==> "ACCURACY": ACCURACY, ==> "IPTS": IPTS, ==> "MUPULSES": MUPULSES, "FSAMP": FSAMP, @@ -215,32 +251,24 @@ def resize_emgfile(emgfile, area=None): - first_idx ) - # Compute PNR and SIL - if rs_emgfile["NUMBER_OF_MUS"] > 0: - # Calculate PNR - to_append = [] - for mu in range(rs_emgfile["NUMBER_OF_MUS"]): - res = compute_pnr( - ipts=rs_emgfile["IPTS"][mu], - mupulses=rs_emgfile["MUPULSES"][mu], - fsamp=emgfile["FSAMP"], - ) - to_append.append(res) - rs_emgfile["PNR"] = pd.DataFrame(to_append) - - # Calculate SIL - to_append = [] - for mu in range(rs_emgfile["NUMBER_OF_MUS"]): - res = compute_sil( - ipts=rs_emgfile["IPTS"][mu], - mupulses=rs_emgfile["MUPULSES"][mu], - ) - to_append.append(res) - rs_emgfile["SIL"] = pd.DataFrame(to_append) - - else: - rs_emgfile["PNR"] = np.nan - rs_emgfile["SIL"] = np.nan + # Compute SIL or leave original ACCURACY + if accuracy == "recalculate": + if rs_emgfile["NUMBER_OF_MUS"] > 0: + if not rs_emgfile["IPTS"].empty: + # Calculate SIL + to_append = [] + for mu in range(rs_emgfile["NUMBER_OF_MUS"]): + res = compute_sil( + ipts=rs_emgfile["IPTS"][mu], + mupulses=rs_emgfile["MUPULSES"][mu], + ) + to_append.append(res) + rs_emgfile["ACCURACY"] = pd.DataFrame(to_append) + + else: + raise ValueError( + "Impossible to calculate ACCURACY (SIL). IPTS not found" + ) return rs_emgfile, start_, end_ @@ -263,10 +291,11 @@ def compute_idr(emgfile): idr : dict A dict containing a pd.DataFrame for each MU (keys are integers). Accessing the key, we have a pd.DataFrame containing: - mupulses: firing sample. - diff_mupulses: delta between consecutive firing samples. - timesec: delta between consecutive firing samples in seconds. - idr: instantaneous discharge rate. + + - mupulses: firing sample. + - diff_mupulses: delta between consecutive firing samples. + - timesec: delta between consecutive firing samples in seconds. + - idr: instantaneous discharge rate. Examples -------- @@ -400,8 +429,7 @@ def delete_mus(emgfile, munumber, if_single_mu="ignore"): "SOURCE" : SOURCE, "RAW_SIGNAL" : RAW_SIGNAL, "REF_SIGNAL" : REF_SIGNAL, - ==> "PNR" : PNR, - ==> "SIL" : SIL + ==> "ACCURACY" : ACCURACY ==> "IPTS" : IPTS, ==> "MUPULSES" : MUPULSES, "FSAMP" : FSAMP, @@ -413,14 +441,10 @@ def delete_mus(emgfile, munumber, if_single_mu="ignore"): """ # Common part working for all the possible inputs to munumber - # Drop PNR values and reset the index - del_emgfile["PNR"] = del_emgfile["PNR"].drop(munumber) + # Drop ACCURACY values and reset the index + del_emgfile["ACCURACY"] = del_emgfile["ACCURACY"].drop(munumber) # .drop() Works with lists and integers - del_emgfile["PNR"] = del_emgfile["PNR"].reset_index(drop=True) - - # Drop SIL values and reset the index - del_emgfile["SIL"] = del_emgfile["SIL"].drop(munumber) - del_emgfile["SIL"] = del_emgfile["SIL"].reset_index(drop=True) + del_emgfile["ACCURACY"] = del_emgfile["ACCURACY"].reset_index(drop=True) # Drop IPTS by columns and rename the columns del_emgfile["IPTS"] = del_emgfile["IPTS"].drop(munumber, axis=1) @@ -458,9 +482,44 @@ def delete_mus(emgfile, munumber, if_single_mu="ignore"): "While calling the delete_mus function, you should pass an integer or a list to munumber= " ) + # Verify if all the MUs have been removed. In that case, restore column + # names in empty pd.DataFrames. + if del_emgfile["NUMBER_OF_MUS"] == 0: + # pd.DataFrame + del_emgfile["IPTS"] = pd.DataFrame(columns=[0]) + del_emgfile["BINARY_MUS_FIRING"] = pd.DataFrame(columns=[0]) + # list of ndarray + del_emgfile["MUPULSES"] = [np.array([])] + return del_emgfile +def delete_empty_mus(emgfile): + """ + Delete all the MUs without firings. + + Parameters + ---------- + emgfile : dict + The dictionary containing the emgfile. + + Returns + ------- + emgfile : dict + The dictionary containing the emgfile without the empty MUs. + """ + + # Find the index of empty MUs + ind = [] + for i, mu in enumerate(range(emgfile["NUMBER_OF_MUS"])): + if len(emgfile["MUPULSES"][mu]) == 0: + ind.append(i) + + emgfile = delete_mus(emgfile, munumber=ind, if_single_mu="remove") + + return emgfile + + def sort_mus(emgfile): """ Sort the MUs in order of recruitment. @@ -489,8 +548,7 @@ def sort_mus(emgfile): "SOURCE" : SOURCE, "RAW_SIGNAL" : RAW_SIGNAL, "REF_SIGNAL" : REF_SIGNAL, - ==> "PNR" : PNR, - ==> "SIL": SIL, + ==> "ACCURACY": ACCURACY, ==> "IPTS" : IPTS, ==> "MUPULSES" : MUPULSES, "FSAMP" : FSAMP, @@ -502,20 +560,20 @@ def sort_mus(emgfile): """ # Identify the sorting_order by the first MUpulse of every MUs - df = pd.DataFrame() - df["firstpulses"] = [ - emgfile["MUPULSES"][i][0] for i in range(emgfile["NUMBER_OF_MUS"]) - ] + df = [] + for mu in range(emgfile["NUMBER_OF_MUS"]): + if len(emgfile["MUPULSES"][mu]) > 0: + df.append(emgfile["MUPULSES"][mu][0]) + else: + df.append(np.inf) + + df = pd.DataFrame(df, columns=["firstpulses"]) df.sort_values(by="firstpulses", inplace=True) sorting_order = list(df.index) - # Sort PNR (single column) - for origpos, newpos in enumerate(sorting_order): - sorted_emgfile["PNR"].loc[origpos] = emgfile["PNR"].loc[newpos] - - # Sort SIL (single column) + # Sort ACCURACY (single column) for origpos, newpos in enumerate(sorting_order): - sorted_emgfile["SIL"].loc[origpos] = emgfile["SIL"].loc[newpos] + sorted_emgfile["ACCURACY"].loc[origpos] = emgfile["ACCURACY"].loc[newpos] # Sort IPTS (multiple columns, sort by columns, then reset columns' name) sorted_emgfile["IPTS"] = sorted_emgfile["IPTS"].reindex(columns=sorting_order) @@ -790,9 +848,9 @@ def get_mvc(emgfile, how="showselect", conversion_val=0): conversion_val : float or int, default 0 The conversion value to multiply the original reference signal. I.e., if the original reference signal is in kilogram (kg) and - conversion_val=9.81, the output will be in Newton/Sec (N/Sec). - If conversion_val=0 (default), the results will simply be Original - measure unit. conversion_val can be any custom int or float. + conversion_val=9.81, the output will be in Newton (N). + If conversion_val=0 (default), the results will simply be in the + original measure unit. conversion_val can be any custom int or float. Returns ------- diff --git a/setup.py b/setup.py index 424f559..af9026c 100644 --- a/setup.py +++ b/setup.py @@ -6,16 +6,16 @@ from pathlib import Path INSTALL_REQUIRES = [ - "customtkinter>=5.1.3", - "matplotlib>=3.7.1", - "numpy>=1.24.3", - "openpyxl>=3.1.2", - "pandas>=2.0.2", - "pandastable>=0.13.1", - "pyperclip>=1.8.2", - "scipy>=1.10.1", - "seaborn>=0.12.2", - "joblib>=1.3.1", + "customtkinter==5.2.0", + "matplotlib==3.7.1", + "numpy==1.25.0", + "openpyxl==3.1.2", + "pandas==2.0.3", + "pandastable==0.13.1", + "pyperclip==1.8.2", + "scipy==1.11.1", + "seaborn==0.12.2", + "joblib==1.3.1", ] PACKAGES = [ @@ -58,7 +58,7 @@ license="GPL-3.0", project_urls={ "Documentation": "https://giacomovalli.com/openhdemg", - "Release Notes": "https://giacomovalli.com/openhdemg/What%27s-New", + "Release Notes": "https://giacomovalli.com/openhdemg/what%27s-new", "Source Code": "https://github.com/GiacomoValliPhD/openhdemg", "Bug Tracker": "https://github.com/GiacomoValliPhD/openhdemg/issues", },