-
Notifications
You must be signed in to change notification settings - Fork 48
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Cylon-RP] Cylon scaling test with radical-pilot (#661)
* [Cylon-RP] Cylon scaling test with radical-pilot Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon-RP] Cylon scaling test with radical-pilot Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Fix panda build error Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Update numpy and panda version Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Update panda version Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon-radical pilot] Fix updated radical pilot script for scaling test Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Fix Cylon CI system and add slice operation on scaling test Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Fix cython version for CI build Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Cylon] Update new version Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * fix native build * [Cylon] Fix numpy and pyarrow dependencies Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * [Pyarrow] Update workspace file for CI build Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * update native script Signed-off-by: Arup Sarker <arupcsedu@gmail.com> * Update build.sh Co-authored-by: niranda perera <niranda.perera@gmail.com> * Update c-cpp.yml --------- Signed-off-by: Arup Sarker <arupcsedu@gmail.com> Co-authored-by: niranda perera <niranda.perera@gmail.com>
- Loading branch information
1 parent
4dcee8d
commit a82e62b
Showing
20 changed files
with
1,103 additions
and
87 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
# Running Cylon on Rivanna | ||
|
||
Arup Sarker (arupcsedu@gmail.com, djy8hg@virginia.edu) | ||
|
||
|
||
|
||
## Install instructions for Radical Pilot | ||
|
||
Rivanna is an HPC system offerbed by University of Virginia. | ||
This will use custom dependencies of the system gcc, openmpi version. | ||
Use the same python environment "cylon_rct" for radical-pilot | ||
|
||
```shell | ||
module load gcc/9.2.0 openmpi/3.1.6 python/3.7.7 cmake/3.23.3 | ||
source $HOME/cylon_rct/bin/activate | ||
pip install radical.pilot | ||
``` | ||
For checking all dependent library version: | ||
|
||
```shell | ||
radical-stack | ||
``` | ||
You need to export mongo-db url: | ||
|
||
```shell | ||
export RADICAL_PILOT_DBURL="mongodb:ADD_YOUR_URL" | ||
``` | ||
Setup is done. Now let's execute scaling with cylon. | ||
|
||
```shell | ||
cd /some_path_to/cylon/rivanna/rp-scripts | ||
python rp_scaling.py | ||
``` | ||
|
||
If you want to make any change in the uva resource file(/some_path_to/radical.pilot/src/radical/pilot/configs) or any other places in the radical pilot source, | ||
|
||
```shell | ||
git clone https://github.com/radical-cybertools/radical.pilot.git | ||
cd radical.pilot | ||
``` | ||
For reflecting those change, you need to upgrade radical-pilot by, | ||
|
||
```shell | ||
pip install . --upgrade | ||
``` | ||
|
||
To uninstall radical pilot, execute | ||
|
||
```shell | ||
pip uninstall radical.pilot | ||
``` |
Oops, something went wrong.