Computational Brain Science Group Server ---------- The Computational Brain Science Group Server is a new, local server system for those with computational neuroscience needs. The server is intended for research use only. There are fees for storage (per TB) and power user accounts, and PI's that are paying for storage can have standard user accounts at no extra cost. ---------- @[toc] ## Setting up an account ### New PI accounts 1. You **must** use your valid uwo.ca email address when requesting an account. 2. Please see below for the current and future fee schedules. 3. It is easy to increase the size of storage allocations. Consider only asking for a nominal amount of storage to start with, as it is not easy to decrease the size of a storage allocation once it has been set up. PIs looking to set up a new account should fill out the following Teams [form][1]. ### New user (non-PI) accounts 1. All user accounts must be associated with a pre-existing PI account. 2. You **must** use your valid uwo.ca email address when requesting an account. 3. Your PI will be charged for each 'power user' account, so please check with him/her to see if you need this functionality. Non-PI users looking to get up a new account should fill out the following Teams [form][2]. ### Account types - Standard user account: 4 cores/8 GB (CBS Basic) - Power user account: 12 cores/32 GB (CBS Heavy) - Power user account: 12 cores/32 GB + GPU (Virtual image under construction) ### Fee schedule Full cost recovery prices **\*** (reflects actual costs of operating the system) 1. Storage - \\$150/T/year 2. First power user account - \\$3000/year for first account, \\$1500/year for each subsequent account Through current CFI and BrainsCAN support, we can offer a 67% cost reduction, such that the charges are: 1. Storage - \\$50/T/year 2. Power user account - \\$1000/year for first account, \\$500/year for each subsequent account Note: With the purchase of storage, any number of Standard user accounts can be created. **\*** Please quote these full prices whenever factoring in costs for the server in grant applications. Billing will be done on a fixed quarterly schedule. ### After you have filled out the form The information you provide in the google form will be forwarded onto Robarts IT, who will in turn set up your account. You will be emailed a password (userid will be the same as your UWO userid) by Suzanne. We attempt to fulfill all new account requests within 1 business day of receiving the request, however, there may be delays if either Suzanne or Robarts IT is busy. **Note:** After you receive your password, it may take an hour or so before all aspects of your account are functional. ---------- ## Account changes Do you need to make changes to your account? Use whichever form is appropriate. 1. PIs needing to increase storage, create new project spaces, or update speedcode, please use this [form][3]. 2. Users (both PI and non-PI) wishing to request access to additional datashares, change account type (Basic or Power), update contract end date, or request access to project spaces, please use this [form][4]. ---------- ## Using the Virtual Desktops The compute server uses a VMware Virtual Desktop Infrastructure (VDI) to instantiate virtual clones of the linux machine each time a user logs in. ### Logging in to the VDI The server is accessed as a virtual machine. 1. Users will need to connect to: <https://vsecurity.robarts.ca> 2. Download and install the VMWare Horizon Client (link near bottom of <https://vsecurity.robarts.ca>). 3. Launch the VMWare Horizon Client software and set up a new connection server. 4. Use ``vsecurity.robarts.ca`` as the connection server address. 5. You should then be prompted to log in with your server username and password. As we are transitioning to using the WTS authentication system, please add ``@robarts.ca`` to your username. 6. You will use the same server username and password to log into Ubuntu. ---------- ### Changing your password From a terminal inside of the VM type: 1. ``kinit [username]`` 2. Enter your password when prompted. 3. ``kpasswd`` 4. Enter your old password when prompted. 5. Enter your new password when prompted. 6. Re-enter your new password when prompted. ### Shutting down vs. disconnecting from virtual machine - If you wish to keep your virutal machine running (e.g., you still have a process running), then either just quit the VMWare Horizon application or select "Disconnect" from the VMWare Horizion application "Connection" pull-down menu. - If you do not have any active processes running, please be considerate of the shared resources and shut down your virutal machine either by selecting "Log Out" from your account menu in Ubuntu or selecting "Log Out" from the VMWare Horizon application "Connection" pull-down menu.  **Note**: While there is no formal policy, the CBS server administrators reserve the right to shut down virtual machines that have been left running but otherwise idle for a significant length of time. Current guidelines will shutdown CBS Basic machines after about 15-30 minutes of inactivity. CBS Heavy machines will be shutdown after 24 hours of inactivity. Leaving an idle virtual machine running takes resources away from other users. ---------- ### Mounting datashares on the VDI #### Using cifs To access a PI datashare via the ```cifs``` mount, use either of the following: 1. To list contents: ``` ls /cifs/<pi_last_name>``` 2. To navigate to datashare: ``` cd /cifs/<pi_last_name>``` If your PI has set up secure project spaces, then use the following commands: 1. Navigate to datashare: ``` cd /cifs/<pi_last_name>/data``` 2. Navigate to project spaces: ``` cd /cifs/<pi_last_name>/projects``` **Note:** Replace ``<pi_last_name>`` with your PI's last name all in lowercase. #### Using mount command As datashare access is tied to a user's credentials, users will need to manually mount datashares each time they start up a VM. To mount your PI's datashare, use the following command: ``mount /srv/<pi_last_name>`` You will be prompted to enter your password. Afterwards, you will see the contents of the datashare in ``/srv/<pi_last_name>``. The datashare will remain mounted until you log out of the VM, shutting it down. If your PI has set up secure project spaces, then you will need to use the following path to access the main datashare: ``mount /srv/<pi_last_name>/data`` and this path to access the secure project spaces: ``mount /srv/<pi_last_name>/projects`` **Note:** Replace ``<pi_last_name>`` with your PI's last name all in lowercase. #### Automatically mounting datashares on VDI at log-in - DEPRECATED - USE THE INSTRUCTIONS FOR CIFS ABOVE Ali has written a script that will allow users to automatically mount their PI's datashare at log-in. The script creates a master key using Passphrase and links it to a user's server credentials. **IF YOU ARE A NEW SERVER USER, USE THE CIFS METHOD DESCRIBED ABOVE FOR MOUNTINT YOUR PI'S DATASHARE. SUPPORT WILL NOT BE GIVEN FOR TROUBLESHOOTING ERRORS WITH THE SETUP_AUTOMOUNT_V2 SCRIPT.** 1. Start up the VDI and log into the Ubuntu VM. 2. Open a terminal and type: ```setup_automount_v2```  3. The script will check if a Passphrase master key already exists. If none is found, the user will be prompted to type ```Y``` to continue creating a master key for the user's account.  4. A Passphrase GUI window will open asking the user to enter a new password for their master key. This can be the same as their server password, or something completely different. **Pick something you will remember.** (Sorry, forgot to screen grab.) 5. After clicking ```OK``` in the password GUI window to accept your master key password and closing the window, the user will be prompted to enter and re-enter their Robarts password (e.g., same password you use to log into the VDI and Ubuntu VM).  6. A GUI window will open up allowing the user to select the datashare directory they would like to have automounted. **Note: users can only automount their PI(s) datashares.**  7. After clicking ```OK``` in the upper right-hand corner of the GUI, the user will be prompted in a second Passphrase GUI to enter the password they created in step 4. Make sure you check the ```Save in password manager.``` box before clicking okay.  8. Users who have access to multiple datashares will need to re-run this script for every datashare they wish to have automounted. ---------- ### Mounting Compute Canada on the VDI Ali has written a script that will set up a sshfs to allow users to mount a Compute Canada machine (e.g., graham) on the VDI. **N.B.:** Users must have an active Compute Canada account before running this script. 1. From your home directory, type: ```setup_sshfs_alias```. 2. Follow the prompts. Users looking to mount graham on the VDI can accept all of the defaults. Users looking to mount a different Compute Canada machine should consult the Compute Canada [wiki][5] to find the relevant system addresses. 3. When prompted to save the ssh keypair, please enter your CBS server password. 4. Once the ssh keypair has been saved, type ```graham``` to mount the Compute Canada system, ```graham```, to your home directory (```/home/ROBARTS/<user>/graham/```). 5. Type ```graham_umount``` to unmount graham. 6. Note: ```graham``` is not set to automount upon log in, so users will need to manually run step 4 to mount ```graham``` at log in and step 5 to unmount ```graham``` when finished. ---------- ### Storage on the VDI This system should only be used to store data and files related to research. It should not be used as a primary backup for lab computers. There is really only one directory in which to store data and files permanently on the server, and this is your PI's datashare (```/srv/<pi_last_name>```). Small text files (e.g., console login information) or python environments may be stored in a user's home directory (```/home/ROBARTS/<username>```), but home directories are not meant to store large files or large numbers of files. **Users are responsible for ensuring that their home directories do not exceed 10GB in size.** To check the size of your home directory at any time, run either: 1. To check overall size: ```du -h /home/ROBARTS/<user>``` 2. To check overall size and list top-level files and directories from largest to smallest: ```du -hd 1 $HOME | sort -h``` **All files and data stored anywhere on the system outside of a PI's datashare or a user's home directory will be deleted when the virtual machine is shut down.** ---------- ### Software on the VDI The VDI system is running Ubuntu 18.04.4 LTS, with a variety of software pre-installed. For those that have not used Ubuntu before, please see [Ubuntu Getting Started](https://help.ubuntu.com/stable/ubuntu-help/getting-started.html). Some software is accessible through the application launcher, but most is available through the command-line, via the `Terminal` application. Users of the VDI do not have admin (sudo) access to install software directly onto the VDI system, but can either install software into a folder they can access (home directory or /srv/<pi_last_name>), or make a software install request. #### Environment modules Most software, wherever possible, will be installed into the `/srv/software` NFS share, and can be added or removed from each user's environment using the `module` command. To show the modules installed: `module list` To unload all modules: `module purge` To list the available modules: `module avail` To load a module: `module load matlab` To load a specific version: `module load matlab/2019b` To get more extensive help on module: `module` User-specific or group-specific module files can also be created, please see the [lmod documentation](https://lmod.readthedocs.io/en/latest/index.html) or contact support-cbs-server@uwo.ca for help. #### Using Python on the VDI Python3 and virtualenv is installed on the server. You can also use Microsoft VS Code (available in launcher), or Pycharm Educational (available as a module), to manage your environment. __Instructions for setting up virtualenv:__ ``` virtualenv ~/myenv source ~/myenv/bin/activate pip install ... ``` Use of Anaconda is discouraged since it does not work well in multi-user environments. If you want your own conda environment, you can install miniconda in your home directory: __Instructions for local install of miniconda:__ ``` wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh sh ./Miniconda3-latest-Linux-x86_64.sh conda init ``` Now after logging out and back in, or `source ~/.bashrc`, you can `conda install ...` or `pip install ...` into your base conda environment #### Software install requests Please e-mail support-cbs-server@uwo.ca for software install requests. ---------- ## Using the storage without VDI (i.e. mounting locally) It is possible to mount your PI's datashare on your local machine. This will allow you to access files and folders on the datashare and use them with applications installed on your local machine. This will also allow you to transfer files between the server and local machines or other mounted drives. Use of the ``tsclient`` is not recommended for transferring data; it is an inherently unstable connection. Mounting a datashare on your local machine is best done from a Western network. When on Western campus, mounting a datashare on your local machine can be done from the wired Western network, uwosecure-v2 wifi, and eduroam wifi. **Note**: - Replace ``<PI_Last_Name>`` with your PI's last name with the first letter capitalized (e.g., Smith). - Replace ``<pi_last_name>`` with your PI's last name all lowercase (e.g., smith). - Replace ``<username>`` with your BMI server username (e.g., jsmith7). This should be the same as your UWO username. ### Mac OS 1. Use 'Connect to Server...' under the 'Go' menu and enter either of the following: - ``cifs://bmisrv.robarts.ca/<PI_Last_Name>_data$`` - ``smb://bmisrv.robarts.ca/<PI_Last_Name>_data$`` 2. If your PI has project spaces, then you can use either of the following to connect: - ``cifs://bmisrv.robarts.ca/<PI_Last_Name>_projects$`` - ``smb://bmisrv.robarts.ca/<PI_Last_Name>_projects$`` 3. When prompted to log in, use the following as your username: ``<username>@robarts.ca`` **Note**: Both ``cifs`` and ``smb`` protocols should work, however, it is preferred to use ``cifs``. ### Windows 1. From Windows Explorer, navigate to: ``\\bmisrv.robarts.ca\<PI_Last_Name>_data$\`` 2. To access a project space, use: ``\\bmisrv.robarts.ca\<PI_Last_Name>_projects$\`` 3. When prompted to log in, use the following as your username: ``<username>@robarts.ca`` ### Linux 1. Make sure you have `cifs-utils` installed on your local Linux machine. 2. Create a folder/directory on local machine (The below example puts the folder in the local machine home folder, however, you can put this folder anywhere.): ``sudo mkdir /home/<pi_last_name>_data`` 3. Open the following file: ``sudo nano /etc/fstab`` 4. Add the following line to the end of the fstab file: ``//bmisrv.robarts.ca/<pi_last_name>_data$ /home/<pi_last_name>_data cifs username=<username>@robarts.ca,noauto,rw,users 0 0`` 5. Run the following command from the terminal: ``mount.cifs //bmisrv.robarts.ca/<pi_last_name>_data$ /home/<pi_last_name>_data`` 6. You will be prompted to enter your server password. 7. If the mounting does not work, make sure that you have your PI's datashare mounted within the VM environment. See the above section, "Automatically mounting datashares on VDI at log-in" on how to mount your PI's datashare within the VM environment. ### Off-campus mounting of datashares Mounting datashares locally is not available from non-Western networks, due to Western ITS regulations of Western network firewalls. To access files stored on CBS server datashares, users will either need to use the Western VPN (Pulse Secure) with the above instructions, or use the VDI or remote desktop into a computer on the Western network and access the datashares from there. **Note: We have discovered that using VPN access does not appear to work for users without Robarts affiliation. We are working to resolve this.** ---------- ## Need help? support-cbs-server@uwo.ca ---------- ## Glossary - **Default user account**: Standard, free user account with access to 4 cores/8 GB - **Power user account**: Fee-based user account with access to 12 cores/32 GB + GPU - **VDI-client**: VMWare Horizon application - **VM**: Ubuntu virutal machine accessed via VMWare Horizon application - **datashares**: Backed-up storage drive tied to a PI's account. [1]: https://forms.office.com/r/dztbCVfNxk [2]: https://forms.office.com/r/mdmWQ6qpep [3]: https://forms.office.com/r/Syb4tKN717 [4]: https://forms.office.com/r/sQepNZDNqS [5]: https://docs.computecanada.ca/wiki/Compute_Canada_Documentation