Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move the network hashes implementation from Bmv2 testgen to lib/ #4526

Merged
merged 15 commits into from
Mar 19, 2024

Conversation

vlstill
Copy link
Contributor

@vlstill vlstill commented Mar 12, 2024

There are hash functions from various network protocols hidden deep in the BMv2 testgen target implementation. This moves then to the lib/ so they can be used elsewhere (mainly I would presume in other testgen targets, but I don't see anything testgen/p4tools specific about them, therefore I'm moving them to top-level lib/).

I'm open to a better name then nethash if someone has a suggestion :-).

@vlstill vlstill added the p4tools Topics related to the P4Tools back end label Mar 12, 2024
@vlstill vlstill requested a review from fruffy March 12, 2024 15:42
@vlstill vlstill self-assigned this Mar 12, 2024
@vlstill vlstill added the run-validation Use this tag to trigger a Validation CI run. label Mar 12, 2024
@vlstill
Copy link
Contributor Author

vlstill commented Mar 12, 2024

I plan to do follow-up PRs with more library functions to allow code sharing with the BMv2 concolic functions implementation.

@vlstill
Copy link
Contributor Author

vlstill commented Mar 12, 2024

Seems like the macOS failure may be related to #4525.

@fruffy
Copy link
Collaborator

fruffy commented Mar 12, 2024

Fyi, these are lifted from https://github.com/p4lang/behavioral-model/blob/5f1c590c7bdb32ababb6d6fe18977cf13ae3b043/include/bm/bm_sim/calculations.h but at this point we can freely modify them. The good thing is changes are differentially tested against BMv2's execution of the same hashes.

lib/nethash.cpp Outdated Show resolved Hide resolved
backends/p4tools/modules/testgen/targets/bmv2/concolic.cpp Outdated Show resolved Hide resolved
backends/p4tools/modules/testgen/targets/bmv2/concolic.cpp Outdated Show resolved Hide resolved
lib/nethash.cpp Outdated

/* generating from my Python script gen_crc_tables inspired from the C code at:
http://www.barrgroup.com/Embedded-Systems/How-To/CRC-Calculation-C-Code */
/* This code was adapted from:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make this comment part of the header?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I think that both this, and the license snipped with (c) Barefoot should be in the .cpp. The reason is that is the code that has that source. The declarations are kind of "on top of that". Or we can have the disclaimer at both, but I think it should be at the implementation and so should be the license/copyright.

lib/nethash.h Show resolved Hide resolved
@vlstill
Copy link
Contributor Author

vlstill commented Mar 15, 2024

@fruffy, I've done a little refactoring on to of the move and I've added basic tests for the CRC sums I've refactored. If you feel like this should be done separately, I can end this at 2f2161c and split the rest.

@vlstill
Copy link
Contributor Author

vlstill commented Mar 15, 2024

The good thing is changes are differentially tested against BMv2's execution of the same hashes.

Sadly it seems at least the crcCCITT variant is never used in existing tests. I've accidentally broken it and only realized it after all the tests passes.

@vlstill vlstill requested a review from fruffy March 15, 2024 15:01
@fruffy
Copy link
Collaborator

fruffy commented Mar 16, 2024

@fruffy, I've done a little refactoring on to of the move and I've added basic tests for the CRC sums I've refactored. If you feel like this should be done separately, I can end this at 2f2161c and split the rest.

Good idea, didn't even consider that we need tests. I think it makes sense to add the tests here: https://github.com/p4lang/behavioral-model/blob/main/tests/test_calculations.cpp

We can do this in this PR.

vlstill added a commit to p4lang/behavioral-model that referenced this pull request Mar 18, 2024
@vlstill
Copy link
Contributor Author

vlstill commented Mar 18, 2024

@fruffy, do you have any code suggestion? Do you think someone else should review this? I would really like to get this one in reasonably soon.

As for the tests in BMv2, I've tried adding them (p4lang/behavioral-model#1235), but I can't compile BMv2 on my work machine and sadly the CI does not show me what failed. I will re-try later.

@fruffy
Copy link
Collaborator

fruffy commented Mar 18, 2024

Oh I mean the other way round. Since this is now a lib class we should copy the BMv2 tests to the compiler. Apologize for the confusion.

Other than that I have no further comments.

@vlstill
Copy link
Contributor Author

vlstill commented Mar 19, 2024

Oh I mean the other way round. Since this is now a lib class we should copy the BMv2 tests to the compiler. Apologize for the confusion.

Oh! There are not many in BMv2. But I've added them and wrote some more for csum16 & identity.

@vlstill vlstill enabled auto-merge March 19, 2024 07:33
@vlstill
Copy link
Contributor Author

vlstill commented Mar 19, 2024

The validate does not build, but this is unrelated -- some problem with RemoveUnusedDeclarations pass.

@vlstill vlstill changed the title [P4Testgen] Move the network hashes implementation from Bmv2 testgen to lib/ Move the network hashes implementation from Bmv2 testgen to lib/ Mar 19, 2024
@vlstill vlstill added the core Topics concerning the core segments of the compiler (frontend, midend, parser) label Mar 19, 2024
@vlstill
Copy link
Contributor Author

vlstill commented Mar 19, 2024

Changed the PR name to indicate that this is not only about testgen.

@fruffy
Copy link
Collaborator

fruffy commented Mar 19, 2024

The validate does not build, but this is unrelated -- some problem with RemoveUnusedDeclarations pass.

Will fix that.

@vlstill vlstill added this pull request to the merge queue Mar 19, 2024
github-merge-queue bot pushed a commit that referenced this pull request Mar 19, 2024
* Move the network hashes implementation from Bmv2 testgen to lib/

* Add missing stream operator signature

* Apply suggestions from code review

Co-authored-by: Fabian Ruffy <5960321+fruffy@users.noreply.github.com>

* Update namespace to CamelCase

* Shuffle the file-level comments and license to make more sense

* Add basic CRC unit tests

* Generalize CRC calculation, avoid duplicates

* Polish the reflection function

* Fix format

* Fix crcCCITT

* Make constructor explicit

* Add comments on the checksums (mainly CRC)

* Add more tests, some from BMv2

---------

Co-authored-by: Fabian Ruffy <5960321+fruffy@users.noreply.github.com>
lib/nethash.h Show resolved Hide resolved
@fruffy fruffy removed this pull request from the merge queue due to a manual request Mar 19, 2024
@vlstill vlstill requested a review from fruffy March 19, 2024 15:43
@vlstill vlstill enabled auto-merge March 19, 2024 15:45
@vlstill vlstill added this pull request to the merge queue Mar 19, 2024
Merged via the queue into p4lang:main with commit 814ff5b Mar 19, 2024
16 of 17 checks passed
@vlstill vlstill deleted the nethash branch March 19, 2024 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Topics concerning the core segments of the compiler (frontend, midend, parser) p4tools Topics related to the P4Tools back end run-validation Use this tag to trigger a Validation CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants