Table of Contents
1. Accessing ICHEC
- 1.1 Where should a new user begin?
- 1.2 I don't have an account. How do I sign up?
- 1.3 I already have an account but I cannot login?
- 1.4 My reactivation link expired - can you help?
- 1.5 I moved institution but I wish to keep my existing username - can you help?
- 1.6 I have an active National HPC Project from Fionn. When will I be able to start using Kay?
- 1.7 I'm a member of a condominium from Fionn. When will I be able to start using Kay?
- 1.8 What mailing lists can I be subscribed to?
- 1.9 What is the overall application process?
- 1.10 How do I connect to an ICHEC system?
- 1.11 Can I connect from home or when traveling?
- 1.12 What is SSH? How does it work?
- 1.13 How do I sftp files between my desktop and ICHEC if I have to use a local ssh proxy?
- 1.14 I have received a request for my login details, what should I do?
- 1.15 I can login with SSH but cannot connect with SFTP?
- 1.16 The project I'm looking for is not listed in the Project Application & Management Portal?
- 1.17 Can I extend a National Service Project?
- 1.18 Can I opt-out of maintenance emails for Kay?
- 2. Compilers and software applications
- 3. Submitting batch jobs
4. ICHEC support
- 4.1 How do I contact the ICHEC helpdesk?
- 4.2 What is my ICHEC username?
- 4.3 I have forgotten my password - can you help?
- 4.4 How do I change my password?
- 4.5 I cannot login to the ICHEC Helpdesk portal but my username and password is definitely correct. Can you help?
- 4.6 How do I acknowledge ICHEC in my work?
- 4.7 What are the opening hours for the ICHEC Helpdesk?
- 5. Common problems
1. Accessing ICHEC
1.1 Where should a new user begin?
New ICHEC users should refer to the Kay support page as a starting point. Documentation and specific topics can be found here. New users should also refer to the Education & Training section for training courses and upcoming workshop schedules.
1.2 I don't have an account. How do I sign up?
1.3 I already have an account but I cannot login?
All existing users of Fionn were required to reactivate their account. If you have not done this, see My reactivation link expired - can you help?.
Most users are now on the new HPC system (Kay). Some Project Investigators (PIs) have not reactivated, so their users are not activated on the system until the PI has reactivated.
If you are trying to login to the Project Application & Management Portal, make sure you are using your ICHEC username and not your email address. If this still doesn't work, see I have forgotten my password - can you help?.
1.4 My reactivation link expired - can you help?
If you are a new user to ICHEC, this does not apply to you. All previous active users on the Fionn system who were transferred to Kay will have received a reactivation link in September and October. The link was only valid for 7 days. If you did not reactivate your account, please contact the ICHEC Helpdesk (email email@example.com) and we will be happy to provide you with a new reactivation link.
You must reactivate your account before you can access the new supercomputer. This process should only take a couple of minutes and involves the following:
- Agreeing to a new Acceptable Usage Policy.
- Indicating mailing list preferences.
- Choosing a new password.
Once you have reactivated your account, you will be able to login to the Project Application & Management Portal and the Helpdesk Portal. If you are a member of active HPC project or condo you will receive a follow up e-mail, usually within 24 hours, on how to access the new HPC system and run jobs.
1.5 I moved institution but I wish to keep my existing username - can you help?
Please email firstname.lastname@example.org with your updated details as soon as possible. It is ICHEC policy that usernames cannot be reused later.
Note: If your email address bounces or if we are unable to contact you, your account will be marked inactive and you will not be able to login to any ICHEC system or submit jobs. After approximately 12 months, your account entry will be anonymised. It is urgent that users with existing data or who wish to keep their account must contact the ICHEC Helpdesk ASAP.
1.6 I have an active National HPC Project from Fionn. When will I be able to start using Kay?
If the PI and its users have reactivated and you have an active project, your account has already been enabled on Kay and you should be able to login and submit jobs
If you are a member of a condominium and your institution has signed up for one on Kay, your account has already been enabled on Kay and you should be able to login and submit jobs using this condominium.
It is essential that all existing Project Investigators (PIs) and Users from the Fionn HPC System reactivate their account (all users received an email in August / September).
1.7 I'm a member of a condominium from Fionn. When will I be able to start using Kay?
At present the following condominiums have agreed to a condominium service on Kay:
- dcu01 (DCU)
- dias01 (DIAS)
- nuig02 (NUI Galway)
- nuim01 (Maynooth University)
- ucd01 (UCD)
- ul01 (UL)
All members of these condos are able to login and submit jobs on the Kay HPC system using the condominium account.
1.8 What mailing lists can I be subscribed to?
The primary mailing lists are:
- email@example.com: This is a low-traffic list for announcements regarding the Centre (optional). Users are asked when they register for an ICHEC account if they wish to receive these emails.
If you wish to receive updates about the Centre, please e-mail us at firstname.lastname@example.org, asking to be subscribed to the ichec-announce mailing list. Likewise, please email us if you do not wish to receive these emails.
- email@example.com: This mailing list is intended to provide essential notifications and information relevant to the operation and services of the ICHEC National HPC infrastructure. All users of this service have been included in order that they receive essential notifications.
1.9 What is the overall application process?
An overview of the application procedure from start to submitting jobs:
- If the PI does not already have an ICHEC account, the PI must first register for an ICHEC account. An ICHEC account can take 24 hours to be set up. The PI will receive an email when their account has been activated.
- A PI submits an application for a National HPC Service Project via the Project Application & Management Portal.
- The PI will receive an email letting them know if the project has been accepted or rejected. The email will let them know the further steps required.
- If the project is accepted, other users who wish to take part in the project (e.g. members of the research group) can login to the Project Application & Management Portal with their ICHEC username and password, and apply to join the project.
- New users to ICHEC who wish to take part in the National Service Project (apart from the PI) who do not have an ICHEC account should register for an ICHEC account. An ICHEC account can take 24 hours to be set up. The user will receive an email when their account has been activated.
- The PI approves users who wish to join the project.
- All new projects have a Start Date associated with them, a member of the ICHEC Systems team will normally activate the project on this date and advise the PI when this has been completed.
- If the user is new to Kay, a member of the ICHEC Systems team will email the user letting them know that their is has been set up and will send further instructions.
- PI and users log in and run jobs.
If the PI or users of the project have any questions about this process, please do not hesitate to contact the ICHEC Helpdesk.
1.10 How do I connect to an ICHEC system?
Once your account has been enabled and you have been granted access to a particular system, you can connect to the system using SSH.
Remember: Your ICHEC username and password is the same for accessing any ICHEC system or portal. This is different to your email address. If you have forgotten your username or password, please contact the ICHEC Helpdesk.
After entering this command, you will be prompted for a password. If you are using Windows, you can use an SSH client such as PuTTY.
You will need SSH access outbound (port 22/tcp) to login to these machines. We have noticed in some institutions you will have to login to a network server with this SSH access first (e.g. DIAS). If you get a "Connection timed out" error, please contact your local network administrator first to ensure you have the appropriate ports open. Failing this, please submit an issue to the ICHEC Helpdesk.
1.11 Can I connect from home or when traveling?
In order to connect to Kay via SSH, our password policy allows SSH access (using password authentication) only from an IP address in our member university's network. Users may configure key-authenticated SSH access that will enable access from the wider internet. Please note that the latter requires that you generate SSH public-private key pairs on another computer other than Kay for security reasons (e.g. ssh-keygen for Linux/Mac, or PuTTYgen and Pageant for Windows; your private key should NOT be stored on Kay). Contact us at the ICHEC Helpdesk if you need help setting this up. Your public key should be added to ~/.ssh/authorized_keys.
If you are currently at home and you cannot add your SSH key, please log a ticket with the ICHEC Helpdesk and we can add the public key for you.
Never share your private key with anyone, not even ICHEC.
1.12 What is SSH? How does it work?
Secure Shell (SSH) is a program to log into another computer over a network, to execute commands in a remote machine, and to move files from one machine to another. It provides strong authentication and secure communications over insecure channels. SSH provides secure X connections and secure forwarding of arbitrary TCP connections.
SSH works by the exchange and verification of information, using public and private keys, to identify hosts and users. The ssh-keygen command creates a directory ~/.ssh and files that contain your authentication information. The public key is stored in ~/.ssh/identity.pub and the private key is stored in ~/.ssh/identity. Share only your public key. Never share your private key! To further protect your private key you should enter a passphrase to encrypt the key when it is stored in the filesystem. This will prevent people from using it even if they gain access to your files.
1.13 How do I sftp files between my desktop and ICHEC if I have to use a local ssh proxy?
If your desktop is on a private network and/or you need to ssh to ICHEC machines via a local ssh proxy then the best way to transfer files directly between your desktop and ICHEC is to use sftp over an ssh tunnel. To create the tunnel use something like:
ssh -f -N -L 2222:kay.ichec.ie:22 firstname.lastname@example.org
Then all connections to port 2222 on your desktop will be forwarded via sshproxy to kay.ichec.ie. So to ftp between ICHEC and your desktop you can simply use:
sftp -oPort=2222 ichec_username@localhost
1.14 I have received a request for my login details, what should I do?
If you receive a request for your login details either by email or telephone you should NOT disclose them regardless of whether the person contacting you purports to be from ICHEC or otherwise as it may be an attempt to steal your login credentials as can be the case with online banking for example. Please ignore the request and report it to us directly.
1.15 I can login with SSH but cannot connect with SFTP?
This problem can occur if you have commands in your .bashrc file which produce output, perhaps an error message which can cause the problem to appear to be intermittent. Such commands should be placed in your .bash_profile file instead where they will not interfere with the SFTP connection process.
1.16 The project I'm looking for is not listed in the Project Application & Management Portal?
Only active projects are listed in the Project Application & Management Portal. If the project is not listed, check with your PI that the project has not met its expiry date.
1.17 Can I extend a National Service Project?
In some cases it may be possible to extend a National Service Project end-date. Users are advised to contact the ICHEC Helpdesk if they need more time.
1.18 Can I opt-out of maintenance emails for Kay?
Emails sent to Kay users are mandatory and every user accepted this when they create the account. We only send important emails to inform users with maintenance updates or critical issues. If you don't want to receive those emails, we can deactivate your account and you'll not receive them anymore. If you wish to close your account, email email@example.com. Emails sent to ichec-announce are optional and you can opt-out of these if you like.Back to top
2. Compilers and software applications
2.1 What languages are available?
Fortran, C, and C++ are available on the ICHEC systems. The commands used to invoke the compilers and/or loaders vary from system to system. For more information, see our Documentation section. A number of scripting and interpreted languages are also available.
2.2 What software applications are available?
2.3 Is help with Makefiles available?
2.4 How do I run Gaussian calculations on Kay?
We have included instructions on the Gaussian page of our software section.Back to top
3. Submitting batch jobs
3.1 What is a batch request?
On the Kay cluster, batch processing is managed by the Slurm Workload Manager. Slurm batch requests (jobs) are shell scripts that provide timing, memory, and processor information. For example scripts, see the Kay support page. Slurm uses squeue -a to check the status, and qdel to delete a batch request. For more information also see Slurm Workload Manager, PBS to Slurm, and Slurm Commands pages.
3.2 How do I submit a job to Kay?
All jobs should be submitted to Kay via the Slurm Manager. Slurm uses sbatch my_script_name to submit a job. A sample script can be found on the Kay support page.
3.3 How can I execute a number of serial runs (task farming)?
A taskfarm utility is in place on Kay you can find details here.
3.4 How can I find out how much resources (CPU core hours) are available to my project?
Use the mybalance command as follows:
username@kay:$ mybalance ======================================================================= Core Hours Allocation Information for account : myproject ======================================================================= Allocated Core Hours : 833333.33 Project Consumed Core Hours : 32153.70 Percentage of Project Consumed : 3.86 =======================================================================
This command will return the number of core hours available to all your projects (in the above example myproject). So for instance, if you wish to run a 48 CPU core job for 24 hours, you will need to ensure that you have a minimum of 24*48=1152 core hours on your project's account.
3.5 How is the order that jobs run in on Kay decided?
The order that jobs run in is decided by the Slurm Workload Manager based on a priority it calculates for each job. The calculation of queue priority is relatively complex in order to deal with long queues, large numbers of jobs and various classes of project. Below is a list of the factors that affect queue priority:
- Jobs gain priority based on the time they wait as an idle job in the queue.
- Fair share targets are applied that balance resource usage over time across both projects and project types.
- An expansion factor is used to modify job priority based on requested walltime. Shorter jobs are given extra priority.
- Only idle jobs gain queue priority. Jobs blocked due to soft or hard limits (see explanation below) can't run and don't gain queue priority over time.
In addition to priority, a backfill mechanism can allow jobs to skip the queue fill space that would otherwise have been unused. This allows smaller jobs with shorter runtimes to maximise our utilisation while larger jobs wait for processors to free up.
Kay has hard and soft limits on the number of jobs and processors a single user can use. The soft limits apply when there is a queue and the considerably higher hard limits apply when there are no jobs waiting to run. When a user exceeds whatever limit is in force all their queued jobs are moved from the queue to a blocked state and are only unblocked when a job completes or the limit is no longer active. Blocked jobs are only checked periodically and not on the regular job scheduling interval.
In addition there is a limit on the number of jobs a user may have queued even when they have not exceeded the hard or soft limits. Additional jobs will be moved to a blocked state until a queued job runs or is cancelled. For further information see the scheduling policies page.Back to top
4. ICHEC support
4.1 How do I contact the ICHEC helpdesk?
The ICHEC Helpdesk and the firstname.lastname@example.org email address are the two main entry points to ICHEC's support teams for registered users. Here you can get help in using the service, find out more about ICHEC or send us your comments.
4.2 What is my ICHEC username?
Your ICHEC username is the same for all ICHEC systems, such as the Project Application & Management Portal and the Helpdesk portal. It is different to our email address. If you have forgotten it, please email email@example.com and we will advise what your username is. It may also be contained in previous correspondence from ICHEC.
4.3 I have forgotten my password - can you help?
Please send an e-mail to firstname.lastname@example.org and a member of the ICHEC Systems team will email you a password reset link. For security reasons this link is only valid for 48 hours. If the link expires and you still require a password reset link, please email email@example.com for a new reset link.
4.4 How do I change my password?
Login to the Project Application & Management Portal and click on the 'Password Change Form' tab.
4.5 I cannot login to the ICHEC Helpdesk portal but my username and password is definitely correct. Can you help?
Please send an e-mail to firstname.lastname@example.org and a member of the ICHEC Systems team will merge your Helpdesk tickets. This usually occurs as you have previously emailed before registering for an account. Remember that you can always log a ticket by email, you do not need to use the Helpdesk portal.
4.6 How do I acknowledge ICHEC in my work?
We would appreciate a formal acknowledgement of ICHEC by inclusion in any resulting publications of the following sentence or some variation thereof: "The authors wish to acknowledge the DJEI/DES/SFI/HEA Irish Centre for High-End Computing (ICHEC) for the provision of computational facilities and support."
It is also important that you notify us of the publication at email@example.com
4.7 What are the opening hours for the ICHEC Helpdesk?
Our User Support Opening Hours are from 09:00 - 17:00 Monday - Friday (Excluding Bank & Public Holidays).Back to top
5. Common problems
5.1 Disk quota exceeded
When projects disk quota is exceeded the PI will received an email notification. You can check the disk usage using the
quota command. Details of how to do so on Kay can be found here.
5.2 How do I remove ^M characters at the end of input lines?
UNIX treats the end of lines differently than other operating systems. Sometimes when editing files in both Windows and UNIX environments, a CTRL-M character gets added to the end of each line as ^M in vi. To remove the ^M characters use the following command:
5.3 How do I run remote X applications?
If you are connecting from a MS Windows machine you will need to have Xming, Hummingbird Exceed or similar installed and running on your workstation. You also need to ensure that X11 forwarding is enabled.
On a typical Unix system X11 support will already be in place and you need connect using:
ssh -X firstname.lastname@example.org
Now you can execute any of your X applications and it should forward the X application via SSH to your local screen. For example:
$ xclock &
You should now see the Xclock is running on your local screen.
5.4 Can I share files with members of my project?
In addition to individual user's home directories e.g. /ichec/home/users/username each project has an associated project directory /ichec/work/project_code which is accessible by all members of that group. By default a user's home directory and its contents cannot be read by other users though you can change the directory permissions to allow this if you wish.
chmod -R 750 directory_name
This command will make the contents of the named directory readable by the other members of you project.
5.5 Do you remind users about my project expiration date?
Yes. You can check when your project is due to expire on the Project Application and Management Portal. You may be eligible to request a project expiration, if you wish to request one you should contact the ICHEC Helpdesk.
- 4 weeks before the project expires, all members of your project will receive an email reminding them of the upcoming project expiration date.
- On the date the project expires, job submission will be automatically disabled. The project members will receive an email reminding them to remove their data from the Kay National HPC System.
- 8 weeks after the project expires, access to the work directory will denied. If the project member does not have any other active projects or condo memberships on Kay, they will be unable to login to Kay. In order to comply with GDPR policy, all project data will be erased: this is an automated process and can't be reverted. As per our Acceptable Usage Policy, project work directories are NOT backed up; make sure to implement a regular backup schedule for your valuable data. Users are strongly advised to backup their data before their project expires.