Update Documentation authored by Roderick Christoph Dedekind's avatar Roderick Christoph Dedekind
...@@ -46,7 +46,7 @@ To gain access to RAMSES you need to fulfill three requirements (in any order): ...@@ -46,7 +46,7 @@ To gain access to RAMSES you need to fulfill three requirements (in any order):
- apply for a project - apply for a project
- secure the connection with ssh keys - secure the connection with ssh keys
- setup a second authentication factor - setup a second authentication factor (2FA)
Apply for a **user account**: Apply for a **user account**:
- [Application form for ITCC projects](https://hpc-access.itcc.uni-koeln.de/jards/WEB/application/login.php?appkind=itcc) - [Application form for ITCC projects](https://hpc-access.itcc.uni-koeln.de/jards/WEB/application/login.php?appkind=itcc)
...@@ -54,9 +54,6 @@ Apply for a **user account**: ...@@ -54,9 +54,6 @@ Apply for a **user account**:
New users can apply for a trial account with limited core/GPU hours without a project description. Applications for a full account need a project description to be reviewed. Up to 15 million core hours per project, a technical review (reasonable usage of resources) is sufficient. Beyond that, a scientific review (importance of research) becomes necessary. New users can apply for a trial account with limited core/GPU hours without a project description. Applications for a full account need a project description to be reviewed. Up to 15 million core hours per project, a technical review (reasonable usage of resources) is sufficient. Beyond that, a scientific review (importance of research) becomes necessary.
### 2FA ### 2FA
For security reasons, you can't login with a username/password. We use a system called **2-Factor-Authentication** (2FA/MFA), meaning you need to prove your identity with two different (as in different systems/locations) 'factors': For security reasons, you can't login with a username/password. We use a system called **2-Factor-Authentication** (2FA/MFA), meaning you need to prove your identity with two different (as in different systems/locations) 'factors':
...@@ -71,7 +68,7 @@ If you own a [Yubikey](https://en.wikipedia.org/wiki/YubiKey) hardware token, yo ...@@ -71,7 +68,7 @@ If you own a [Yubikey](https://en.wikipedia.org/wiki/YubiKey) hardware token, yo
Please note: we can't provide Yubikeys to users, but it could be a worthwhile investment for about 50€. Please note: we can't provide Yubikeys to users, but it could be a worthwhile investment for about 50€.
After you have successfully enrolled in Duo and prepared your SSH Key, please After you have successfully enrolled in Duo and prepared your SSH Key, please
send your key to pabel@uni-koeln.de . send your key.
### Generate SSH keys<a name="ssh-gen"></a> ### Generate SSH keys<a name="ssh-gen"></a>
...@@ -119,7 +116,7 @@ cat ~/.ssh/id_ed25519.pub ...@@ -119,7 +116,7 @@ cat ~/.ssh/id_ed25519.pub
``` ```
If your `ssh` on your computer is old, it will not know the key type ed25519. If `ssh` on your computer is old, it will not know the key type ed25519.
In this case use In this case use
``` ```
...@@ -157,14 +154,14 @@ password. ...@@ -157,14 +154,14 @@ password.
If you have to use a Windows System: [Key-based authentication in OpenSSH for Windows](https://learn.microsoft.com/en-gb/windows-server/administration/openssh/openssh_keymanagement) If you have to use a Windows System: [Key-based authentication in OpenSSH for Windows](https://learn.microsoft.com/en-gb/windows-server/administration/openssh/openssh_keymanagement)
If you already have access to RAMSES but you are using the CHEOPS key, I If you already have access to RAMSES but you are using the CHEOPS key, we
advise you to create your own SSH key on your local machine/laptop and then advise you to create your own SSH key on your local machine/laptop and then
add the public key to your `.ssh/authorized_keys` file in your home on RAMSES. add the public key to your `.ssh/authorized_keys` file in your home on RAMSES.
Any text editor will work for this. Any text editor will work for this.
**PLEASE NOTE**: Do no share SSH Keys with other people and do not copy around private keys to other computers. Just create new SSH Key pairs on each computer you use regularly. You can also use SSH Agent Forwarding, where an SSH Key is taken along into a SSH session to a remote computer, eliminating the need to create many keys. **PLEASE NOTE**: Do no share SSH Keys with other people and do not copy private keys to other computers. Just create new SSH Key pairs on each computer you use regularly. You can also use [SSH Agent Forwarding](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/using-ssh-agent-forwarding), where an SSH Key is taken along into a SSH session to a remote computer, eliminating the need to create many keys.
Once you received your access credentials and set up your SSH keys and Cisco Duo, you have to send us the **public** part of your keypair. (*.pub). **AGAIN:NEVER EVER SHARE THE PRIVATE KEY** Once you received your access credentials and set up your SSH keys and Cisco Duo App(s), you have to send us the **public** part of your keypair. (*.pub). **AGAIN:NEVER EVER SHARE THE PRIVATE KEY**
Please send the public key to: [hpc-mgr@uni-koeln.de](mailto:hpc-mgr@uni-koeln.de) Please send the public key to: [hpc-mgr@uni-koeln.de](mailto:hpc-mgr@uni-koeln.de)
### LOGIN<a name="login"></a> ### LOGIN<a name="login"></a>
...@@ -174,7 +171,7 @@ ramses1.itcc.uni-koeln.de up to ramses4.itcc.uni-koeln.de ...@@ -174,7 +171,7 @@ ramses1.itcc.uni-koeln.de up to ramses4.itcc.uni-koeln.de
Do not use ramses2 or ramses3, they are for internal use only for now. Do not use ramses2 or ramses3, they are for internal use only for now.
When logging in to ramses1, the public key you sent us is authenticated with the private key on your computer (1st factor, you will be asked for the ssh passphrase, see also [here](#ssh-usage)). If successful, a verification request is automatically pushed to the Duo App on your device where you confirm the login (2nd factor). When logging in to ramses1, the public key you sent us is authenticated with the private key on your computer (1st factor, you will be asked for the ssh passphrase). If successful, a verification request is automatically pushed to the Duo App on your device where you confirm the login (2nd factor).
On your terminal you should see something like this: On your terminal you should see something like this:
...@@ -219,27 +216,27 @@ an admin). ...@@ -219,27 +216,27 @@ an admin).
## Data transfer<a name="data-transfer"></a> ## Data transfer<a name="data-transfer"></a>
To transfer your data to the cluster, we recommend using scp (**s**ecure **c**o**p**y) - either on the command line (CLI/Terminal) or with a graphical client (e.g. WinSCP).\ To transfer your data to the cluster, we recommend using [scp](https://tldr.inbrowser.app/pages/common/scp) (**s**ecure **c**o**p**y) - either on the command line (CLI/Terminal) or with a graphical client (e.g. WinSCP).\
There is no automatic mechanism to sync/copy files between Cheops and Ramses. You have to copy your files yourself. There is no automatic mechanism to sync/copy files between Cheops and Ramses. You have to copy your files yourself.
Please note: you can transfer data ONLY to the login nodes (ramses1 ... ramses4), NEVER directly to compute nodes. Please note: you can transfer data ONLY to the login nodes (ramses1 ... ramses4), not directly to compute nodes.
- for small numbers of files/folder: - for small numbers of files/folder:
```one or more computation steps, each with one or multiple (parallel) tasks and specific resources (cores, nodes, RAM etc.) ```
- scp local_file username@ramses1.itcc.uni-koeln.de:remote_destination_dir ( . for home folder) - scp local_file username@ramses1.itcc.uni-koeln.de:remote_destination_dir ( . for home folder)
- scp -r local_folder username@ramses1.itcc.uni-koeln.de:remote_destination_dir - scp -r local_folder username@ramses1.itcc.uni-koeln.de:remote_destination_dir
``` ```
- for huge amounts of small files use [tar](https://www.gnu.org/software/tar/manual/html_chapter/Tutorial.html#Tutorial) (or zip) to create an archive-file before copying: - for huge amounts of (small) files use [tar](https://www.gnu.org/software/tar/manual/html_chapter/Tutorial.html#Tutorial) (or zip) to create an archive-file before copying:
``` ```
- create: tar -czf name_of_archive.tar.gz files_or_folder_to_add - create: tar -czf name_of_archive.tar.gz files_or_folder_to_add
- extract: tar -xvzf example.tar.gz - extract: tar -xvzf example.tar.gz
- show contents: tar -tf <file> - show contents: tar -tf <file>
``` ```
- you can also use rsync -\> see: [rsync](man.rsync) - you can also use [rsync](https://tldr.inbrowser.app/pages/common/rsync)
- if you prefere interactive transfer with a shiny GUI: e.g. [FileZilla (Linux/Mac/Win)](https://filezilla-project.org/), [WinSCP (Win only)](https://winscp.net/eng/download.php), [Cyberduck (Mac only)](https://cyberduck.io/download/) - if you prefer interactive transfer with a shiny GUI: e.g. [FileZilla (Linux/Mac/Win)](https://filezilla-project.org/), [WinSCP (Win only)](https://winscp.net/eng/download.php), [Cyberduck (Mac only)](https://cyberduck.io/download/)
<br> <br>
<br> <br>
...@@ -260,7 +257,6 @@ The filesystem setup is exactly as on CHEOPS: ...@@ -260,7 +257,6 @@ The filesystem setup is exactly as on CHEOPS:
- NO AUTOMATIC BACKUP, automatic deletion of files will be activated soon - NO AUTOMATIC BACKUP, automatic deletion of files will be activated soon
- typical usage: input data should be copied to the scratch-partition only for running or soon running jobs. Accordingly, input and temporary data on /scratch should be deleted and output data transferred to longer term storage after job completion. - typical usage: input data should be copied to the scratch-partition only for running or soon running jobs. Accordingly, input and temporary data on /scratch should be deleted and output data transferred to longer term storage after job completion.
- /project/\<user/group\> - /project/\<user/group\>
- size per user ...
- created on request - created on request
- NO AUTOMATIC BACKUP - NO AUTOMATIC BACKUP
... ...
......