Performance

[ISK-VAGR]

Hi,

I am kind of new on Github. I was testing the PPScore and I really like it. However, when I try to do datasets over 1000 correlations it takes forever. I am having an M1 mac and I see that neither the M1 GPU nor my 20 cores are used at their full potential. Any idea of how to improve this?

I already tried "numexpr" and n_jobs, multiprocessing, and ProcessPoolExecutor. I got short on ideas to improve its performance. I will appreciate help on this.

Thanks,

AG

[8080labs]

Where did you pass in the multiprocessing and n_jobs etc? It is true that the current implementation is not optimized for speed but using a single thread/processor.Sent from mobileOn 20 Jan 2023, at 14:15, 'ISK-VAGR' via Admin ***@***.***> wrote: Hi, I am kind of new on Github. I was testing the PPScore and I really like it. However, when I try to do datasets over 1000 correlations it takes forever. I am having an M1 mac and I see that neither the M1 GPU nor my 20 cores are used at their full potential. Any idea of how to improve this? I already tried "numexpr" and n_jobs, multiprocessing, and ProcessPoolExecutor. I got short on ideas to improve its performance. I will appreciate help on this. Thanks, AG —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ISK-VAGR]

What I meant is that non of the above speed options worked when I added them to my code. I was checking the cores while doing the run and it was with 4 or 5 cores by default. I ignore the reason for this. I also know that apple is especially difficult in this regard. I have a powerful Mac but I was never been able to leverage it. The results of your code are really impressive though. The goal standard for me will be to do it for about 8000 correlations, but at this speed, I don't think if it will be ready in a week.

[8080labs]

Can you share the code that you ran?Sent from mobileOn 20 Jan 2023, at 18:25, 'Dr. Vladimir Andrey Gimenez Rivera' via Admin ***@***.***> wrote: What I meant is that non of the above speed options worked when I added them to my code. I was checking the cores while doing the run and it was with 4 or 5 cores by default. I ignore the reason for this. I also know that apple is especially difficult in this regard. I have a powerful Mac but I was never been able to leverage it. The results of your code are really impressive though. The goal standard for me will be to do it for about 8000 correlations, but at this speed, I don't think if it will be ready in a week. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: ***@***.***>