Skip to content

Add operator to spatial transform ts4231 data #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Add operator to spatial transform ts4231 data #432

wants to merge 13 commits into from

Conversation

cjsha
Copy link
Member

@cjsha cjsha commented Apr 16, 2025

Resolve #427

@cjsha cjsha requested a review from bparks13 April 16, 2025 13:29
@cjsha cjsha requested a review from jonnew April 16, 2025 13:33
Copy link
Member

@bparks13 bparks13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good, I did not test the functionality but did leave some comments on some potential restructuring of the code, and the need to add XML comments before we can merge this.


private void EnableButtons()
{
buttonMeasure0.Invoke((Action)delegate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are the EnableButtons and DisableButtons so different? I would have thought you could create a single method SetButtonState(bool state) and assign the boolean to each button

Copy link
Member Author

@cjsha cjsha Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so I was getting "cross-thread operation not valid" errors during run-time when I would call win forms methods from a Subscribe's lambda function. Using this winFormControl.Invoke((Action) delegate {...}) pattern (relevant documentation) was one way to get around this. There might be another more elegant to prevent this error though. I wanted to look into it before issuing the PR, but I think it became time to put this out there and see what people think. Did you ever run into this kind of error?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never encountered this error, but I also was not running the dialogs while the workflow was active. Is this the method that the other Bonsai nodes use to handle passing data?

Copy link
Member Author

@cjsha cjsha Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really good question. When I started, I cloned the Bonsai library and started developing on top of their GroupBy operator, slowly incorporating in bits of pieces from the RoiActivity operator and my own code to make it what we need. When I was doing this, I never ran into these errors. When I ported it to the OpenEphys.Onix1 library, these errors started occurring. I'm not immediately sure how these threads are being handled under the hood that this behavior wouldn't occur when developing on top of the Bonsai library but they would in our library.

Copy link
Member Author

@cjsha cjsha Apr 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can get around using the winFormControl.Invoke((Action) delegate {...}) pattern by using the ObserveOn and ControlScheduler. This is probably how Bonsai takes care of this problem. I'm using their Bonsai.Design.ControlScheduler() method to do this. This is what I added in c3e951d

cjsha added 2 commits April 16, 2025 16:07
- Consolidate win form event handlers
- Initialize SpatialTransformMatrix property to indentity matrix
- Refactor some functions here and there
- This is a more elegant than using the `WinForm.Control.Invoke` pattern to avoid cross-threading errors
@cjsha cjsha added this to the 0.5.0 milestone Apr 18, 2025
- I think the the sptial transform should be another property within
  TS4231PositionData that allows the user to map the base-station-based
  coordinate system into an external coordinate system
- Defaulting to identity matrix means that this does nothing by default.
Copy link
Member

@jonnew jonnew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting idea but we need to do some things to make it useful generally. I've committed some changes that start to move in this direction. Other refinements are

  1. The user needs to be able to cancel a measurement and start over. Or a timeout needs to be set that will cancel a measurement if some time has elapsed before lighthouse data is received.
  2. Ideally the editor should not display unless the workflow is running. I think there are ways to check the run state of a workflow when an editor is opened.

- add cancel button
- add timeout (probably only need timeout or cancel button)
- check workflow is running before opening GUI
- leave text boxes blank
- correctly populate ts4231V1CoordinatesMatrix
- give persistent scope to subscriptions so they can be disposed
- simplify conditional statement for checking if user input is valid
- minor edit to top-level label to be consistent with changes
@cjsha cjsha requested a review from jonnew April 23, 2025 00:04
@cjsha
Copy link
Member Author

cjsha commented Apr 23, 2025

I made changes and did some cursory testing to make sure it looks ok and still works. I didn't have time to extensively test this latest commit.

There is one problem with the current implementation which is that the calculated matrix spatial transform is only correct if we are already starting with an identity matrix.

@jonnew
Copy link
Member

jonnew commented May 6, 2025

It seems like the user supplied coordinates are verified in real-time for the correct format. It would be nice to have a status text indicating if the input is valid or not (e.g. "input coordinates invalid vs. input coordinates valid". You can a tool strip for this (see e.g. Rhs2116StimulusSequenceDialog.cs:

image

@jonnew
Copy link
Member

jonnew commented May 6, 2025

Instead of this:

image

use the standard "OK, Cancel" pattern:

image

See Rhs2116StimulusSequenceEditor for an example.

…Dialog

- This should recover the raw position values from P and Q alone without
  the M that is currently being edited.
Copy link
Member

@jonnew jonnew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there. I added some comments and I think i took care of the issue with non-default values of M.

Please do some verification with real headstage in known space and wacky values for initial value of M.

@cjsha
Copy link
Member Author

cjsha commented May 7, 2025

@jonnew Re:

It would be nice to have a status text indicating if the input is valid or not (e.g. "input coordinates invalid vs. input coordinates valid". You can a tool strip for this (see e.g. Rhs2116StimulusSequenceDialog.cs

I'm wondering if it would be worth indicating which coordinate is invalid. like having green checks or red x-es for each row or each cell (probably instead of the status strip) as indicated by this screenshot:

Screenshot 2025-05-07 084942

jonnew's feedback:
- Add status strip
- Use "OK", "Cancel" button paradigms

additionally:
- Improve resizing
- Address all VS messages
  - PascalCase methods
  - Make "using" syntax more concise
  - Add/Edit some XML comments
- Instead of transforming every Vector3 and then averaging, average all Vector3s and then take the transform
- Inline/remove the GetData() function
@cjsha
Copy link
Member Author

cjsha commented May 20, 2025

I addressed feedback (look at commit message for more details). I tested calculating a new M from a pre-set M. This seems to work OK.

@cjsha cjsha requested a review from jonnew May 20, 2025 00:02
@ChucklesOnGitHub ChucklesOnGitHub self-assigned this May 22, 2025
@jonnew
Copy link
Member

jonnew commented May 30, 2025

If I click measure for a coordinate with data in it, and the measurement is invalid, it doesnt get rid of the old measurement

image

Desired?

@jonnew
Copy link
Member

jonnew commented May 30, 2025

Here are some changes I think would be helpful to the UI

image

  • In addition, please put more line breaks between items in the instructions. Its very compact and hard to read.
  • Also, make sure Tab works as expected for the entered values (goes in row major order)
    image

@jonnew
Copy link
Member

jonnew commented May 30, 2025

It is possible to create a transform that collapses dimensions by providing zero for a given dimension. For instance if all my calibration data is in a single X,Y plane, this is the result:

image

This should not be permitted.

@jonnew
Copy link
Member

jonnew commented May 30, 2025

Several overall architectural comments:

  • The property M should be replaced with a custom class that holds both M and all the measurements and corresponding user coordinates used to create it. This way, when the GUI is invariably reopened, the user can make incremental edits instead of needing to start over.
  • The need to invert M to reopen the GUI needs to go away. To do this, I suggest that we turn this into a transform called TS4231SpatialTransform. TS4231PositionData remains the uncalibrated position data its always been. A new, EmbeddedWorkflow is created called TS4231CalibratedPositionData that combines TS4231PositionData with the TS4231SpatialTransform. This way the transform always has access to untransformed data an inversion of M is not required.

Copy link
Member

@jonnew jonnew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please have a look at my comments in the PR

@jonnew jonnew modified the milestones: 0.5.0, 0.5.1 Jun 2, 2025
cjsha added 5 commits June 11, 2025 15:57
- The operator is now an included workflow comprising of ts4231
  source node and a spatial transform node
- The property that's set by the dialog is a struct containing
  pre-transform coordinates, post-transform coordinates and spatial
  transform matrix instead of just the matrix
- Add workflows to ItemGroup in Onix1.csproj
- Revert TS4231V1PositionData
- Dialog changes:
  - Add X, Y, Z labels
  - Add a textbox for each component of each coordinate instead of
    one textbox
  - Add a textbox & label for displaying Spatial Transform Matrix
  - Change status messages TextBox to RichTextBox which allows
    changing font color and using newline characters instead of
    environment.newline.
  - Automatically calculate transform matrix when inputs are valid
    (avoids decoupling sets of pre-transform & post-transform
    coordinates and the spatial transform)
  - Simplify bottom toolstrip behavior
  - Move event handlers to top and helper methods underneath
  - Change instructions in top label according to the above changes
@ChucklesOnGitHub ChucklesOnGitHub requested a review from jonnew June 23, 2025 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow users to apply a spatial transform matrix to ts4231 data in the data operator
4 participants