hpc:cgi

Creating GPU Instances

Create only GPU instances

$ sudo nvidia-smi mig -cgi 14,14,19,19,20
Successfully created GPU instance ID  5 on GPU  0 using profile MIG 2g.20gb (ID 14)
Successfully created GPU instance ID  3 on GPU  0 using profile MIG 2g.20gb (ID 14)
Successfully created GPU instance ID 13 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created GPU instance ID  9 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created GPU instance ID 10 on GPU  0 using profile MIG 1g.10gb+me (ID 20)

Create GPU instances and apply default compute instance profile

$ sudo nvidia-smi mig -cgi 14,14,19,19,20 -C
Successfully created GPU instance ID  5 on GPU  0 using profile MIG 2g.20gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  5 using profile MIG 2g.20gb (ID  1)
Successfully created GPU instance ID  3 on GPU  0 using profile MIG 2g.20gb (ID 14)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  3 using profile MIG 2g.20gb (ID  1)
Successfully created GPU instance ID 13 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 13 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID  9 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  9 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID 10 on GPU  0 using profile MIG 1g.10gb+me (ID 20)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 10 using profile MIG 1g.10gb (ID  0)

One big GPU

$ sudo nvidia-smi mig -cgi 0 -C
Successfully created GPU instance ID  0 on GPU  0 using profile MIG 7g.80gb (ID  0)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  0 using profile MIG 7g.80gb (ID  4)

$ nvidia-smi -L
GPU 0: NVIDIA A100 80GB PCIe (UUID: GPU-a166d193-64ba-8969-91b0-7729983c680b)
  MIG 7g.80gb     Device  0: (UUID: MIG-c4f17314-2b46-5569-ac07-bb0781b44ba7)
  
$ nvidia-smi
Thu Mar 10 21:34:25 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100 80G...  Off  | 00000000:06:00.0 Off |                   On |
| N/A   32C    P0    67W / 300W |      0MiB / 81920MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    0   0   0  |      0MiB / 81069MiB | 98      0 |  7   0    5    1    1 |
|                  |      1MiB / 13107... |           |                       |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Many (7) smallest GPUs

~$ sudo nvidia-smi mig -cgi 19,19,19,19,19,19,19 -C
Successfully created GPU instance ID 13 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 13 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID 11 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 11 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID 12 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 12 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID  7 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  7 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID  8 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  8 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID  9 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  9 using profile MIG 1g.10gb (ID  0)
Successfully created GPU instance ID 10 on GPU  0 using profile MIG 1g.10gb (ID 19)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID 10 using profile MIG 1g.10gb (ID  0)

$ nvidia-smi -L
GPU 0: NVIDIA A100 80GB PCIe (UUID: GPU-a166d193-64ba-8969-91b0-7729983c680b)
  MIG 1g.10gb     Device  0: (UUID: MIG-6655c8fd-3878-5030-a9b0-65b40839f65e)
  MIG 1g.10gb     Device  1: (UUID: MIG-52c663bb-eef5-580c-b98f-2c2d93f4d668)
  MIG 1g.10gb     Device  2: (UUID: MIG-efafa9b6-09cc-5b9f-9a43-73dc927794f0)
  MIG 1g.10gb     Device  3: (UUID: MIG-3f2ece72-3e45-560d-8698-79d951b1312b)
  MIG 1g.10gb     Device  4: (UUID: MIG-09f3602a-747a-5357-9d60-fd883da6fae2)
  MIG 1g.10gb     Device  5: (UUID: MIG-38d561b9-0ddf-5033-9fc0-1016588a3854)
  MIG 1g.10gb     Device  6: (UUID: MIG-33709a1d-5135-5f0d-a6ab-b936857ed5e4)

$ nvidia-smi
Thu Mar 10 21:36:19 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100 80G...  Off  | 00000000:06:00.0 Off |                   On |
| N/A   32C    P0    68W / 300W |     45MiB / 81920MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    7   0   0  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0    8   0   1  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0    9   0   2  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0   10   0   3  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0   11   0   4  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0   12   0   5  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0   13   0   6  |      6MiB /  9728MiB | 14      0 |  1   0    0    0    0 |
|                  |      0MiB / 16383MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

One big GPU with many compute instances

$sudo nvidia-smi mig -cci 0
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)

$ sudo nvidia-smi mig -cci 0,0,0,0,0,0,0
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  1 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  2 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  3 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  4 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  5 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)
Successfully created compute instance ID  6 on GPU  0 GPU instance ID  0 using profile MIG 1c.7g.80gb (ID  0)

$ nvidia-smi -L
GPU 0: NVIDIA A100 80GB PCIe (UUID: GPU-a166d193-64ba-8969-91b0-7729983c680b)
  MIG 1c.7g.80gb  Device  0: (UUID: MIG-c4f17314-2b46-5569-ac07-bb0781b44ba7)
  MIG 1c.7g.80gb  Device  1: (UUID: MIG-2b5e73e1-66c1-5801-8084-ef0d0ec08b6c)
  MIG 1c.7g.80gb  Device  2: (UUID: MIG-000028a5-5c26-532f-b903-38ffc3ed1422)
  MIG 1c.7g.80gb  Device  3: (UUID: MIG-3046d0e8-aaf9-546e-a9a7-5673d31eeea8)
  MIG 1c.7g.80gb  Device  4: (UUID: MIG-80c0ecbd-29bb-5944-8576-a0c04b437a27)
  MIG 1c.7g.80gb  Device  5: (UUID: MIG-e5a655a5-9430-51ef-abca-3f8c1527023c)
  MIG 1c.7g.80gb  Device  6: (UUID: MIG-b6de6961-47b2-56c1-88f9-26775c22e1a7)
$ nvidia-smi
Thu Mar 10 21:21:23 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100 80G...  Off  | 00000000:06:00.0 Off |                   On |
| N/A   32C    P0    61W / 300W |      0MiB / 81920MiB |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    0   0   0  |      0MiB / 81069MiB | 14      0 |  7   0    5    1    1 |
|                  |      1MiB / 13107... |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   1   1  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   2   2  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   3   3  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   4   4  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   5   5  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+                      +-----------+-----------------------+
|  0    0   6   6  |                      | 14      0 |  7   0    5    1    1 |
|                  |                      |           |                       |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+