Caching harmonic sums #202

felixhekhorn · 2023-01-26T16:14:31Z

Here is my proposal for a central harmonic sums cache. Main concepts:

we drag one cache array around which is not worse than dragging the sx array around (and better than the two in Cache harmonics sums for the as2 anomalous dimensions #179 ); furthermore this would solve Removing deprecated list of list in harmonic cache #143 , since we resort to a plain list
we fill this array slowly whenever we request a new harmonic sum
and so the moment we compute a new harmonic sum, we register it in the cache
the central task for everybody that needs a harmonic sum is to register a "key" (in cache.py) (Note that the order does not matter, since we would never use the number of the key, but only its name)
when computing the actual harmonic sum we even could go recursive and have e.g. $g_3$ and $S_{-2,1}$ simultaneously around (we can either go recursive or just do a straight return there, á là if key == Sm21: return get(g3)+n+zeta, or we inline the relation where the thing is needed)

Please tell me what you think!

The single commit here is just to lay out the idea at LO and by no means complete ...

cc @enocera

TODO

move harmonic constants to global constants

giacomomagni · 2023-01-27T09:42:42Z

Hi @felixhekhorn thanks for this idea, looks good to me! I have two minor comments:

We should add another argument is_singlet or eta needed for more complex harmonics
We should define a naming convention for S(1,N/2), S(1, (N-1)/2) ... and similar

src/ekore/harmonics/cache.py

Co-authored-by: Giacomo Magni <39065935+giacomomagni@users.noreply.github.com>

alecandido · 2023-01-27T12:51:24Z

@felixhekhorn are you sure that Numba has no issue with globals?

EDIT: in particular https://numba.pydata.org/numba-doc/dev/user/faq.html#numba-doesn-t-seem-to-care-when-i-modify-a-global-variable

felixhekhorn · 2023-01-27T14:25:39Z

S1 = $S_1(N)$
S1h = $S(1,N/2)$
S1mh = $S(1, (N-1)/2)$
Sm1 = $S(-1,N)$

…and added harmonic constants to eko.constants module

t7phy · 2023-01-30T21:26:18Z

Hi,

I have added all the harmonic sums to the cache that are currently implemented in the w files. I have however not yet added the harmonic sums with N/2 and (N-1)/2 as I was not sure up to which order they need to be added. (let me know which ones are needed and I could update them)
I have also added the harmonic constants to eko.constants module and updated the corresponding imports.
I also added the bool parameter is_singlet to the cache function to allow consistency with the Smx functions.

Perhaps have a look and if its alright, we could merge this also and use the new cache in the splitting functions.

felixhekhorn · 2023-02-09T15:42:33Z

doc/source/overview/features.rst

@@ -1,7 +1,12 @@
 Features
 ========

+<<<<<<< HEAD


now you have some nice merge conflicts - that you need to resolve ;-) (and might it be that you're still not running pre-commit? because that should have prevented you from committing thing ...)

I have been using pre commit for the time_like branch but not here because pydocs requires imperative statements in docstrings(for instance, it needs to be Compute xyz instead of Computes xyz), however a lot of the previous functions (such as in as2, as3, etc.) do not implement it so I just skip pre commit for this branch for the time being. Will get to it soon enough ;)

I know - but for new code you should, of course, do it and then you are only allowed to skip pre-commit in the very last step, i.e. when no one, but pydocstyle is complaining

right, noted!

t7phy · 2023-02-10T01:57:53Z

@felixhekhorn @niclaurenti @giacomomagni @alecandido I have modified the complete space-like implementation to be compatible with the new harmonic cache (ADs (ASx and AEMx) and OMEs). Please have a look.

In principal it should be fine to merge in of itself, however as all the tests were based on old cache system, they are failing. I have updated many of them but there is just too many for me to update on my own.

giacomomagni · 2023-02-10T09:07:01Z

src/eko/evolution_operator/operator_matrix_element.py

+        -------
+        ome : numpy.ndarray
+            matching operator matrix
+    <<<<<<< HEAD:src/eko/evolution_operator/operator_matrix_element.py


here there is an unsolved conflict. The change coming from master is the correct one (same below)

giacomomagni · 2023-02-10T09:13:32Z

src/ekore/anomalous_dimensions/unpolarized/space_like/as2.py



 @nb.njit(cache=True)
-def gamma_nsm(n, nf, sx):
+def gamma_nsm(n, nf, cache, is_singlet):


I think is_singlet should not appear as argument in any function except for nsp which can be used as singlet and as non singlet.
Calling this function with is_singlet=True is misleading, so we should not have this possibility.
Viceversa hold for gg, gq, qg, ps ...
@niclaurenti is the expert for the mixed QED QCD one.

I agree with @giacomomagni . In this case you don't need to pass is_singlet because you already know that we are not in the singlet sector. It is better to call get with is_singlet=False (however notice that is_singlet is useless for the first harmonic sums so consider using a default argument like is_singlet=None)

alecandido · 2023-02-10T09:32:58Z

What is actually missing from this PR? I.e. why is this still draft?

Please, put checkboxes in the OP for all the items, and tick those that you believe to be completed, even if they can be improved (everything can always be improved).
In general, we should try to merge smaller PR, to reduce the amount of changes and improve the review process.

giacomomagni · 2023-04-28T13:04:44Z

Given that #148 and QED have been merged, it's a good time to take care of this.
@t7phy if it's okay for you I can work a bit on this ?

t7phy · 2023-04-28T13:06:01Z

Given that #148 has been merged, it's a good time to take care of this.
@t7phy if it's okay for you I can work a bit on this ?

Yes, please go ahead.

giacomomagni · 2023-04-28T13:20:11Z

okay there are endless confilcts in basically all files... maybe it's easier to start from scratch?

niclaurenti · 2023-04-28T13:24:54Z

Given that #148 and QED have been merged, it's a good time to take care of this.
@t7phy if it's okay for you I can work a bit on this ?

If you need help with the harmonic sums needed in the QED sector I can help

giacomomagni · 2023-04-28T13:29:32Z

If you need help with the harmonic sums needed in the QED sector I can help

I was trying to merge master here again, but everything (almost all the files in ekore display conflicts), we have changed
Time like, QED and N3LO in the meantime + all the refactors...
I'd say we can start again in a new branch. The two main ideas were:

We move all the constants in to eko.constants.
We introduce the harmonic cache as done here.

Was there anything else to save from here @felixhekhorn @t7phy ?

t7phy · 2023-04-28T13:38:40Z

If you need help with the harmonic sums needed in the QED sector I can help

I was trying to merge master here again, but everything (almost all the files in ekore display conflicts), we have changed
Time like, QED and N3LO in the meantime + all the refactors...
I'd say we can start again in a new branch. The two main ideas were:

We move all the constants in to eko.constants.

We introduce the harmonic cache as done here.

Was there anything else to save from here @felixhekhorn @t7phy ?

Yes I agree, I think starting in new branch is simpler. The two files are the only main changes to copy. Ofcourse, you could also copy all the space like ADs and OMEs as I had already added the new harmonic sums to them.

felixhekhorn · 2023-04-28T13:48:34Z

I was trying to merge master here again, but everything (almost all the files in ekore display conflicts), we have changed Time like, QED and N3LO in the meantime + all the refactors... I'd say we can start again in a new branch. The two main ideas were:
1. We move all the constants in to `eko.constants`.

2. We introduce the harmonic cache as done here.
Was there anything else to save from here @felixhekhorn @t7phy ?

I agree, a clean restart might be better ... (solving the is_singlet along the way)

giacomomagni · 2023-04-28T14:01:04Z

closed in favor of #252 and new PR

felixhekhorn and others added 13 commits January 11, 2023 12:50

Split math of status quo

c14b43e

Move ads to new layout

85c2da6

Add pytest coverage

10d2240

fix init name

8fd5436

Start reshuffling OME

4678da7

Move ad benchmark

694a239

Move OME

ec04696

Add missing init file

febd415

propagate polarized and time-like from op card to op.__init__

bb6af22

Add switch for OME

3db8696

Fix function names

7a74ddb

Fix runcard translation

1126089

Init cache at LO

9226ccc

felixhekhorn added the refactor Refactor code label Jan 26, 2023

felixhekhorn requested review from alecandido, giacomomagni, adrianneschauss, niclaurenti and t7phy January 26, 2023 16:14

giacomomagni reviewed Jan 27, 2023

View reviewed changes

src/ekore/harmonics/cache.py Outdated Show resolved Hide resolved

Update src/ekore/harmonics/cache.py

45b95a5

Co-authored-by: Giacomo Magni <39065935+giacomomagni@users.noreply.github.com>

niclaurenti mentioned this pull request Jan 30, 2023

Cache harmonics sums for the as2 anomalous dimensions #179

Closed

felixhekhorn mentioned this pull request Jan 30, 2023

Reorganize package content #194

Merged

alecandido force-pushed the split-math-in-module branch from 4c72a96 to 07aa331 Compare January 30, 2023 16:09

Base automatically changed from split-math-in-module to master January 30, 2023 16:25

added all harmonic sums that are currently implemented to new cache, …

764d876

…and added harmonic constants to eko.constants module

t7phy added 4 commits February 9, 2023 03:12

sm21 and pylint error fixes for cache file

65fee8a

additional cache error fixes

0d2c8d2

merge master in harmonic-sums-cache

51d8ec8

merge master in harmonic-sums-cache

0cf143d

felixhekhorn commented Feb 9, 2023

View reviewed changes

resolve merge conflicts

5de8a74

giacomomagni mentioned this pull request Feb 9, 2023

bye bye harmonics weight 5 #210

Merged

t7phy added 5 commits February 9, 2023 22:18

resolve merge conflicts

07eec29

complete implementation of new harmonic sums cachefor spacelike case

12fbe7a

some error fixes

1f4e9b5

new cache consistency fixes

1e3adf9

fixed some tests to work with new cache

483107c

Merge branch 'master' into harmonic-sums-cache

0d5fd14

giacomomagni reviewed Feb 10, 2023

View reviewed changes

felixhekhorn mentioned this pull request Feb 16, 2023

N3LO ad variations #148

Merged

felixhekhorn mentioned this pull request Mar 28, 2023

Time Like Splitting Functions #232

Merged

giacomomagni closed this Apr 28, 2023

felixhekhorn mentioned this pull request Apr 28, 2023

Refactor harmonic cache #253

Merged

4 tasks

giacomomagni deleted the harmonic-sums-cache branch May 3, 2023 11:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Caching harmonic sums #202

Caching harmonic sums #202

felixhekhorn commented Jan 26, 2023 •

edited by t7phy

Loading

giacomomagni commented Jan 27, 2023 •

edited

Loading

alecandido commented Jan 27, 2023 •

edited

Loading

felixhekhorn commented Jan 27, 2023

t7phy commented Jan 30, 2023 •

edited

Loading

felixhekhorn Feb 9, 2023

t7phy Feb 9, 2023

felixhekhorn Feb 9, 2023

t7phy Feb 9, 2023

t7phy commented Feb 10, 2023 •

edited

Loading

giacomomagni Feb 10, 2023 •

edited

Loading

giacomomagni Feb 10, 2023 •

edited

Loading

niclaurenti Feb 10, 2023

alecandido commented Feb 10, 2023

giacomomagni commented Apr 28, 2023 •

edited

Loading

t7phy commented Apr 28, 2023

giacomomagni commented Apr 28, 2023

niclaurenti commented Apr 28, 2023

giacomomagni commented Apr 28, 2023 •

edited

Loading

t7phy commented Apr 28, 2023

felixhekhorn commented Apr 28, 2023

giacomomagni commented Apr 28, 2023

Caching harmonic sums #202

Caching harmonic sums #202

Conversation

felixhekhorn commented Jan 26, 2023 • edited by t7phy Loading

giacomomagni commented Jan 27, 2023 • edited Loading

alecandido commented Jan 27, 2023 • edited Loading

felixhekhorn commented Jan 27, 2023

t7phy commented Jan 30, 2023 • edited Loading

felixhekhorn Feb 9, 2023

Choose a reason for hiding this comment

t7phy Feb 9, 2023

Choose a reason for hiding this comment

felixhekhorn Feb 9, 2023

Choose a reason for hiding this comment

t7phy Feb 9, 2023

Choose a reason for hiding this comment

t7phy commented Feb 10, 2023 • edited Loading

giacomomagni Feb 10, 2023 • edited Loading

Choose a reason for hiding this comment

giacomomagni Feb 10, 2023 • edited Loading

Choose a reason for hiding this comment

niclaurenti Feb 10, 2023

Choose a reason for hiding this comment

alecandido commented Feb 10, 2023

giacomomagni commented Apr 28, 2023 • edited Loading

t7phy commented Apr 28, 2023

giacomomagni commented Apr 28, 2023

niclaurenti commented Apr 28, 2023

giacomomagni commented Apr 28, 2023 • edited Loading

t7phy commented Apr 28, 2023

felixhekhorn commented Apr 28, 2023

giacomomagni commented Apr 28, 2023

felixhekhorn commented Jan 26, 2023 •

edited by t7phy

Loading

giacomomagni commented Jan 27, 2023 •

edited

Loading

alecandido commented Jan 27, 2023 •

edited

Loading

t7phy commented Jan 30, 2023 •

edited

Loading

t7phy commented Feb 10, 2023 •

edited

Loading

giacomomagni Feb 10, 2023 •

edited

Loading

giacomomagni Feb 10, 2023 •

edited

Loading

giacomomagni commented Apr 28, 2023 •

edited

Loading

giacomomagni commented Apr 28, 2023 •

edited

Loading