Posts Cluster Notes
Post
Cancel

Cluster Notes

MSI

  1. Jupyter Notebook
  2. FileZilla to transfer data files.
  3. Cuda and GPU number setting

Use the same configuration as in CSE Labs, configure ssh to login without password.

Question to ask MSI staff

  1. How to configure the environment for Jupyter notebook? Can I start Jupyter notebook in one environment?

CSE Labs

  1. Set ssh to login without password. Do this on the local machine. Run the following step by step exactly. Check the reference for more detail.
    • Don’t change id_rsa to other names. Don’t enter passphrase. Just press Enter for three times.
      1
      
        ssh-keygen -t rsa
      
    • Even if there are .ssh in the cluster, run it.
      1
      
        ssh x500@csecluster mkdir -p .ssh
      
    • 1
      
       cat .ssh/id_rsa.pub | ssh x500@csecluster 'cat >> .ssh/authorized_keys'
      
    • Now I can log in without a password.
      1
      
        ssh x500@csecluster
      
  2. Set ssh to switch among the clusters without a password. Do this on the cluster machine.
    • Login to one cluster first.
    • Run the following code inside the cluster.
      1
      2
      
        ssh-keygen -t rsa -P ""
        cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
      
  3. Set ssh aliases. Do this on the local machine. Check the reference for more detail.
    • Edit the ~/.ssh/config file. Can also do for phixx and cudaxx account.
      1
      2
      3
      4
      5
      6
      
        # CSElabs Lind account
        Host cse-lind
            HostName CSEcluster
            User x500
            IdentityFile ~/.ssh/id_rsa
            UseKeychain yes
      
    • Then we can directly connect to the cluster using the following.
      1
      
        ssh cse-lind
      
  4. Set ssh aliases among the terminals. Do this on the cluster machine.
    • Create and edit the ~/.ssh/config file.
      1
      2
      3
      4
      
        Host phi01
            HostName phi01.cselabs.umn.edu
            User yang6367
            IdentityFile ~/.ssh/id_rsa
      
    • Then we can switch among the clusters. For example, inside the lind machine, switch to phi01 machine.
      1
      
        ssh phi01
      
  5. Modify the prompt format. Copy git-completion.bash and git-prompt.sh files to the cluster machine. Then add the following to the .bashrc file.
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    
     # Enable tab completion
     source ~/git-completion.bash
    
     # colors!
     green="\[\033[0;32m\]"
     blue="\[\033[0;34m\]"
     purple="\[\033[0;35m\]"
     reset="\[\033[0m\]"
    
     # Change command prompt
     source ~/git-prompt.sh
     export GIT_PS1_SHOWDIRTYSTATE=1
     # '\u' adds the name of the current user to the prompt
     # '\$(__git_ps1)' adds git-related stuff
     # '\W' adds the name of the current directory
     export PS1="$purple\u@$green\h\$(__git_ps1)$blue \W $ $reset"
    
  6. My modules are obsolete, and could not load hpc/openmpi module. After asking Prof. Saad for help, I got the solution. Edit ~/.bashrc file. Change the MODULESINIT as the following.
    1
    
     MODULESINIT="/soft/rko-modules/tcl/init/bash"
    
Updated Feb 23, 2020 2020-02-23T15:14:05-06:00
This post is licensed under CC BY 4.0