Proxmox

From Indie IT Wiki

HOWTO: First Steps After Installation

Add the no subscription repository... 

echo "deb http://download.proxmox.com/debian/pve buster pve-no-subscription" >>/etc/apt/sources.list
apt-get update
apt-get check
apt-get --simulate dist-upgrade

https://bendellar.com/simple-repository-setup-for-proxmox/

HOWTO: Update PCI and USB IDs

update-pciids
update-usbids

HOWTO: Set System Locale

dpkg-reconfigure locales   (and choose en_GB.utf8)

Thanks - https://wiki.debian.org/Locale

HOWTO: Change To Software RAID and LVM

Do NOT do this if you are using the new 4.x version of Proxmox and have chosen ZFS (RAID) filesystem!

Install Software

aptitude install mdadm

Prepare Disks

sgdisk -R=/dev/sdb /dev/sda
sgdisk -t 2:fd00 /dev/sdb
sgdisk -t 3:fd00 /dev/sdb
sgdisk -R=/dev/sdc /dev/sda
sgdisk -t 2:fd00 /dev/sdc
sgdisk -t 3:fd00 /dev/sdc
parted

GNU Parted 2.3
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model: ATA WDC WD3000FYYZ-0 (scsi)
Disk /dev/sda: 3001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
 1      1049kB  2097kB  1049kB               primary  bios_grub
 2      2097kB  537MB   535MB   ext3         primary  boot
 3      537MB   3001GB  3000GB               primary  lvm
(parted)                                                                  
(parted) select /dev/sdb                                                  
Using /dev/sdb
(parted) print                                                  
Model: ATA WDC WD3000FYYZ-0 (scsi)
Disk /dev/sdb: 3001GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number  Start   End     Size    File system  Name     Flags
 1      1049kB  2097kB  1049kB               primary  bios_grub
 2      2097kB  537MB   535MB                primary  raid
 3      537MB   3001GB  3000GB               primary  raid

Create RAID Arrays

mdadm --create /dev/md0 --level=1 --raid-disks=2 missing /dev/sdb2
mdadm --create /dev/md1 --level=1 --raid-disks=2 missing /dev/sdb3
cat /proc/partitions /proc/mdstat

Copy Boot Files To RAID Array

mkfs.ext3 /dev/md0 
mkdir /mnt/tmp
mount /dev/md0 /mnt/tmp
cp -ax /boot/* /mnt/tmp
umount /mnt/tmp
rmdir /mnt/tmp

Change Boot Filesystem Tab

nano /etc/fstab
/dev/pve/root / ext3 errors=remount-ro 0 1
/dev/pve/data /var/lib/vz ext3 defaults 0 1
/dev/md0 /boot ext3 defaults 0 1
#UUID=46b4d3d6-fdec-43b6-a4cb-3f8f8a9c6c10 /boot ext3 defaults 0 1
/dev/pve/swap none swap sw 0 0
proc /proc proc defaults 0 0

Do not reboot now as per kbDone's page!

Update GRUB Bootloader

echo 'GRUB_DISABLE_LINUX_UUID=true' >> /etc/default/grub
echo 'GRUB_PRELOAD_MODULES="raid dmraid"' >> /etc/default/grub
echo raid1 >> /etc/modules
echo raid1 >> /etc/initramfs-tools/modules
grub-install /dev/sda
grub-install /dev/sdb
grub-install /dev/sdc   (because we will actually be adding sdc below)
update-grub
update-initramfs -u

Now reboot. 

reboot

Prepare First Disk (Part 1)

sgdisk -t 2:fd00 /dev/sda
mdadm --add /dev/md0 /dev/sda2

Move LVM Data

pvcreate /dev/md1
vgextend pve /dev/md1
pvmove /dev/sda3 /dev/md1   <-- this takes a very long time!
vgreduce pve /dev/sda3
pvremove /dev/sda3

Prepare First Disk (Part 2)

sgdisk -t 3:fd00 /dev/sda
mdadm --add /dev/md1 /dev/sda3   <-- this takes a long time as well
cat /proc/mdstat

Wait for RAID to sync.

Add Extra (Spare) Disks As Required

mdadm --add /dev/md0 /dev/sdc2
mdadm --add /dev/md1 /dev/sdc3

Update GRUB (Again)

grub-install /dev/sda
grub-install /dev/sdb
grub-install /dev/sdc
update-grub
update-initramfs -t -u
sync

Finish

reboot
cat /proc/mdstat

Thanks - http://kbdone.com/proxmox-ve-3-2-software-raid/

HOWTO: Add ISO Files

In the GUI... 

Storage View > Datacenter > server > local > content > Upload

Location on disk... 

/var/lib/vz/template/iso

HOWTO: Add Virtual Machine (VM)

Server View > Datacenter > server
Create VM

Ubuntu Linux

Copy Existing - Physical To Virtual (P2V)

Physical

Virtual

STOP PRESS - new method?! - http://www.fsarchiver.org/QuickStart

  • Boot with SystemRescueCD iso in virtual cdrom
  • either use the proxmox console, or set root password and restart ssh, and ssh in
  • Partition and Format virtual hard disk drive (1:swap,2:ext4)
  • mkdir /mnt/sda2 and mount -v /dev/sda2 /mnt/sda2/
  • scp physical:/backup.tar.gz root@virtual:/mnt/sda2/
  • sudo tar -xzvpf /mnt/sda2/backup.tar.gz -C /mnt/sda2/ --numeric-owner (close console window for this)
  • mount -t proc none /mnt/sda2/proc
  • mount -o bind /dev /mnt/sda2/dev
  • mount -t sysfs /sys /mnt/sda2/sys
  • chroot /mnt/sda2 /bin/bash
  • blkid /dev/sda1 /dev/sda2
  • nano /etc/mtab /etc/fstab /etc/hosts /etc/network/interfaces (change IP address to different to physical)
  • touch /var/lock/aptitude
  • aptitude -y -v purge mdadm lm-sensors hddtemp smartmontools cpufrequtils
  • rm -rfv /var/lib/smartmontools/
  • grub-install /dev/sda
  • update-initramfs -u
  • update-grub
  • exit
  • reboot

Finish

  • When you are happy, then shut down the Physical and change the Virtual IP address, then reboot Virtual :-)

Windows 7

Fresh Installation

  • Create an ISO of the Windows 7 DVD
dd if=/dev/sr0 of=/var/lib/vz/template/iso/win7pro-sp1.iso
  • Download the VirtIO Drivers ISO (Changelog)
  • Create VM according to the Windows 7 Best Practices page
  • CDROM #1 = win7pro-sp1.iso
  • CDROM #2 = virtio-win-0.1-81.iso
  • CDROM #3 = VMware-tools-windows-9.4.11-2400950.iso
  • Start VM and read instructions below

Mouse Driver

This gives you much better mouse performance and uses less resources (than the default 'usb tablet mouse').

The vmmouse driver need a reboot to get activated but then you can enable/disable the 'usb device tablet' on the fly.

  • extract setup.exe from iso
  • extract datas from setup.exe
  • setup.exe /A /P C:\Extract
  • drivers are in C:\extract\Program Files\VMware\VMware Tools\VMware\Drivers\mouse

Mouse Driver (64 Bit) - http://packages.vmware.com/tools/esx/latest/windows/x64/index.html (e.g. VMware-tools-windows-9.4.11-2400950.iso)

Mouse Driver (32 Bit) - http://packages.vmware.com/tools/esx/latest/windows/x86/index.html (e.g. VMware-tools-windows-9.4.11-2400950.iso)

Qemu Agent

The qemu-guest-agent is a helper daemon, which is installed in the guest. It is used to exchange information between the host and guest, and to execute command in the guest.

This is a must have!

http://pve.proxmox.com/wiki/Qemu-guest-agent

Links

Tweaks - https://pve.proxmox.com/wiki/Performance_Tweaks

Thanks - https://pve.proxmox.com/wiki/Windows_7_guest_best_practices

Thanks - https://pve.proxmox.com/wiki/Windows_VirtIO_Drivers#Choose_the_right_driver

HOWTO: RUN VIRTUAL MACHINE IN PROXMOX INSIDE VIRTUALBOX

Because VirtualBox does not support KVM Stacking or KVM-in-KVM, you have to use the QEMU option in Proxmox when running inside VirtualBox.

  1. Install Proxmox VE inside VirtualBox and start Proxmox
  2. Log in to the Proxmox Web Admin and 'Create VM'
  3. VM > Options > KVM Hardware Virtualization = NO, QEMU Agent = YES

HOWTO: RESIZE DISKS

https://pve.proxmox.com/wiki/Resize_disks

Shut Down Stop The Virtual Machine

Shut down and Stop the VM using the GUI or command line.

GUI

Server View > Datacenter > Node > VM > Hardware > Hard Disk > Resize

COMMAND LINE

In this example, we will be increasing the first virtual hard disk sda from 20Gb to 68Gb.

Resize Virtual Hard Disk

Find out the name of the virtual machine and hard disk, using the GUI (shown above), for example 105 and sata0.

Now, you can resize the virtual disk... 

qm resize 105 sata0 +48G

Resize Hard Disk Partition

Boot the VM with a live CD such as SystemRescueCD, and then use 'parted' to resize the partition (in our case partition #2)... 

parted /dev/sda
(parted) print
(parted) resizepart 2 100%
(parted) print
(parted) quit

Check Hard Disk Filesystem (I) 

e2fsck -f /dev/sda2

Resize Hard Disk Filesystem 

resize2fs /dev/sda2

Check Hard Disk Filesystem (II) 

e2fsck -f /dev/sda2

Poweroff The Virtual Machine

Now, you can shutdown the SystemRescueCD... 

poweroff

START THE VIRTUAL MACHINE

Start the VM using the GUI or command line, login and check your disk size. 

df -H

http://pve.proxmox.com/wiki/Resize_disks

HOWTO: FIX

apt update: TASK ERROR: command 'apt-get update' failed: exit code 100

To fix, run the following option to apt-get ... 

apt-get update --allow-releaseinfo-change

OR 

apt-get update

and answer questions manually ... 

N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.
Do you want to accept these changes and continue updating from this repository? [y/N] y

apt-get update: server certificate verification failed

https://pve.proxmox.com/wiki/Server_certificate_verification_failed_when_updating

NFS Share Unmount When Rebooting Server

Manually

To check mountpoint of NFS share... 

mount

Disable the NFS share (either in web GUI or on the command line)... 

pvesm set yournfsharename_inwebgui --disable 1

To unmount (be patient for up to 10 minutes)... 

fusermount -uz /path/to/mountpoint

If no success, then... 

umount -f -l -vvv /path/to/mountpoint

https://forum.proxmox.com/threads/nfs-share-umount-f-problem-device-is-busy.26777/

Automatically 

nano /etc/rc.local.shutdown

#!/bin/sh -e
echo "NFS Shares are unmounting..."
fusermount -uz /mnt/pve/*
exit 0

http://unix.stackexchange.com/questions/98510/run-a-script-at-shutdown-in-debain#98523

Shutdown Error - TASK ERROR: VM quit powerdown failed - got timeout

Install the app 'acpid' in each of your Linux virtual machines... 

sudo apt install acpid

HOWTO: ENABLE

Add CPU Frequency Scaling

aptitude install cpufrequtils

Thanks - https://wiki.debian.org/HowTo/CpuFrequencyScaling

HOWTO: CONVERT:

From Proxmox RAW Image To VirtualBox Qcow

-rw-r--r-- 1 root root  20G 2015-02-18 12:54 disk-drive-sata0.raw
-rw-r--r-- 1 root root  330 2015-02-18 12:51 qemu-server.conf

qemu-img convert -f raw -O qcow disk-drive-sata0.raw disk-drive-sata0.qcow

-rw-r--r-- 1 root root 9.8G 2015-02-20 11:03 disk-drive-sata0.qcow
-rw-r--r-- 1 root root  20G 2015-02-18 12:54 disk-drive-sata0.raw
-rw-r--r-- 1 root root  330 2015-02-18 12:51 qemu-server.conf

OVA IMAGE TO Qcow

Instructions

This example shows how to convert the ESET Remote Administrator Virtual Appliance to a Proxmox Virtual Machine. 

sudo -i
tar -xf era_appliance.ova
qemu-img convert -p -f vmdk ERA_Appliance-disk1.vmdk -O qcow2 ERA_Appliance-disk1.qcow2
qemu-img info ERA_Appliance-disk1.qcow2

image: ERA_Appliance-disk1.qcow2
virtual size: 64G (68719476736 bytes)
disk size: 7.5G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: false
    corrupt: false

Open up the OVF file in your editor (it's XML format) and just check what hardware it requires.

Then, create your new VM in the Proxmox web GUI as per hardware requirements.

Then, copy over the new QCOW2 disk image file we converted OVER the one created by Proxmox. 

cp -av ERA_Appliance-disk1.qcow2 /var/lib/vz/images/106/vm-106-disk-1.qcow2

Then, edit the Proxmox VM config file to change the disk size to match the figure in bold above. 

nano /etc/pve/nodes/proxmox1/qemu-server/106.conf

bootdisk: ide0
cores: 4
ide0: local:106/vm-106-disk-1.qcow2,format=qcow2,size=64G
ide2: none,media=cdrom
memory: 4096
name: eset1
net0: e1000=5E:EF:04:01:CA:45,bridge=vmbr0
numa: 0
onboot: 1
ostype: l26
smbios1: uuid=12936a8c-6417-4e5d-9f7d-f6e06730b72a
sockets: 1
startup: order=7,up=70,down=10

Now, you can go back to the Proxmox web GUI and start the new VM in a Console.

Job, done.

Troubleshooting

If your VM will not start because of this error:- 

file system may not support O_DIRECT

It is because you are using the new ZFS with Proxmox, which does not support it.

Then change the Cache setting on your VM to Write Through... 

cache=writethrough

Thanks - http://www.jamescoyle.net/how-to/471-zfs-and-glusterfs-network-storage

Links

http://www.jamescoyle.net/how-to/1218-upload-ova-to-proxmox-kvm

http://forum.proxmox.com/threads/9309-Convert-OVA-to-working-Proxmox-VM

http://opentox.github.io/installation/2012/08/02/converting-ova-images-to-kvm/

http://matthewmidgett.com/virtualization/5-converting-virtualbox-to-proxmox

http://edoceo.com/notabene/ova-to-vmdk-to-qcow2

HOWTO: Command Line Tools

https://pve.proxmox.com/wiki/Command_line_tools

Check Storage

Settings are kept in... 

/etc/pve/storage.cfg

Check the current status with... 

pvesm status

This also works to fix an unmounted NFS share (say the NFS server rebooted).

Add A User Using Domain Based Authentication

pveum aclmod / -user joebloggs@DOMAIN -role PVEAuditor

Shell Script To List Status Of Virtual Machines

#!/bin/bash
cd /etc/pve/nodes/<nodename>/qemu-server/
for FILE in *.conf
do
  VMID="${FILE%%.*}"
  echo -n "vm $VMID "
  /usr/sbin/qm status $VMID;
done

Backup All VMs

vzdump --quiet 1 --mailto backupnotify@domain.com --mode snapshot --compress lzo --storage nfs1 --node proxmox1 --all 1

Backup 1 VM

vzdump 219 --mode stop --compress lzo --node proxmox2 --remove 0 --storage nfs1 --mailto backupnotify@domain.com

Shutdown VM

qm shutdown VMID

Thanks - https://pve.proxmox.com/wiki/Virtual_Machine_Startup_and_Shutdown_Behavior

List VMs

qm list

Start VM

qm start VMID

Shutdown VM

qm shutdown VMID

Stop VM

qm stop VMID

Stop Unresponsive Locked VM

qm unlock VMID
qm stop VMID

If that fails then kill the Linux process, which you can find out with ... 

ps -auwx | grep VMID
kill PID

HOWTO: DHCP For VMs On Different IP Address Range

https://jay.sh/setting-up-dhcp-for-proxmox-vms/

HOWTO: Change Cron Scheduled Backup Time

sudo nano /etc/pve/vzdump.cron

HOWTO: Change Node IP to DHCP

$ nano /etc/network/interfaces
auto lo
iface lo inet loopback
iface eno1 inet manual
iface eno2 inet manual
auto vmbr0
iface vmbr0 inet dhcp
  bridge_ports eno1
  bridge_stp off
  bridge_fd 0

HOWTO: Change Node IP Address

Use the Web GUI... 

Server View > Datacenter > Node > Network > vmbr0 > IP Address > OK > Restart

If you are not able to directly access the Web GUI, use SSH to Port Forward to your computer... 

ssh root@proxmox-node -L local-port:remote-server-ip:remote-port

ssh root@123.456.789.0 -L 8006:192.168.1.200:8006

Then, point your web browser to the following address, and hey presto, Web GUI! 

https://127.0.0.1:8006

OLD

Update the IP address and DNS IP address in these files... 

nano /etc/hosts /etc/network/interfaces /etc/resolv.conf

If it's a cluster, run this command as well... 

/etc/init.d/cman restart

Then, reboot... 

reboot

HOWTO: Change Node Hostname Name

WARNING: THIS WORKED FOR ME IN A TEST VIRTUALBOX ENVIRONMENT BUT HAS NOT BEEN TRIED IN PRODUCTION

YOU HAVE BEEN WARNED

THESE INSTRUCTIONS ARE A WORK IN PROGRESS

You may also have to edit some VM .conf files. 

nano /etc/hostname /etc/hosts /etc/postfix/main.cf
cd /etc/pve/nodes/
cp -av old new
rm -rfv old
cd /var/lib/rrdcached/db/pve2-node/
cp -av old new
rm -rfv old
reboot

HOWTO: System Update

https://pve.proxmox.com/wiki/Downloads

  • Change DNS Server to modem
nano /etc/resolv.conf
nameserver 192.168.0.1
  • Shut down all running Virtual Machines
  • Turn off Start at Boot for each Virtual Machine (using WebGUI) or...
nano /etc/pve/nodes/proxmox1/qemu-server/*.conf
onboot: 1    <-- delete this line
  • Update system software...
screen -rd
apt-get update
apt-get check
apt-get --download-only dist-upgrade
apt-get --simulate dist-upgrade
apt-get --quiet dist-upgrade
(answer YES to Continuing Without Installing GRUB)
grub-mkdevicemap -n
grub-install /dev/sda
grub-install /dev/sdb
grub-install /dev/sdc
update-grub
update-initramfs -t -u
sync
touch /root/misc/system_updated
(exit screen)
reboot

HOWTO: Test ZFS With Virtualbox

...



HOWTO: Cluster

https://www.mattwall.co.uk/2019/06/09/new-proxmox-cluster-pt1.html

https://www.howtoforge.com/tutorial/how-to-configure-a-proxmox-ve-4-multi-node-cluster/

http://www.unixmen.com/setting-cluster-proxmox/

HOWTO: Help

Wiki

https://pve.proxmox.com/wiki/Main_Page

Forum

http://forum.proxmox.com/

Retrieved from ‘https://wiki.indie-it.com/index.php?title=Proxmox&oldid=3073’