Installation
June 11, 2019 · View on GitHub
1 Introduction
This document serves as a guide specifying the steps and configuration required for OpenStack installation on servers configured by SNAPS-OpenStack. It does not provide implementation level details.
This document is to be used by development team, validation and network test teams.
1.1 Terms and Conventions
The terms and typographical conventions used in this document are listed and explained in below table.
| Convention | Usage |
|---|---|
| Host Machines | Machines to be used for Openstack deployment. Openstack node controller, compute, storage and network node will be deployed on these machines. |
| Build Server | Machine running installation scripts, Ansible, Python, NTP etc. |
1.2 Acronyms
The acronyms expanded below are fundamental to the information in this document.
| Acronym | Explanation |
|---|---|
| SNAPS | SDN/NFV Application Platform/Stack |
| IP | Internet Protocol |
| COTS | Commercial Off The Shelf |
| OS | OpenStack |
| VLAN | Virtual Local Area Network |
1.3 References
[1] OpenStack Installation guide: https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html
1.4 OpenStack services support under QUEENS release
Basic OpenStack Services • Nova • Neutron • Glance • Keystone • Horizon • Heat
Additional services:
• Tacker • Mistral • Magnum • Barbican • Ceilometer • gnocchi • redis • Cinder • Ceph • SRIOV • DPDK • Designate
1.5 OpenStack IPv6 support under QUEENS release
OpenStack Queens release supports IPV6 functionality for OpenStack(Current support is available only for VM to VM networking)
2 Environment Prerequisites
2.1 Hardware Requirements
The current release of SNAPS-OpenStack is tested on the following platform.
Compute Node
| Hardware Required | Description | Configuration |
|---|---|---|
| Server machine with 64bit Intel AMD architecture. | COTS servers. | 16GB RAM, 80+ GB Hard disk with 3 network cards. Server should be network boot Enabled and IPMI capable. |
Controller Node
| Hardware Required | Description | Configuration |
|---|---|---|
| Server machine with 64bit Intel AMD architecture. | COTS servers. | 16GB RAM, 80+ GB Hard disk with 3 network cards. Server should be network boot Enabled and IPMI capable |
Build Server
| Hardware Required | Description | Configuration |
|---|---|---|
| Server machine with 64bit Intel AMD architecture. | COTS servers. | 16GB RAM, 80+ GB Hard disk with 1 network cards. |
2.2 Software Requirements
| Category | Software version |
|---|---|
| Operating System | Ubuntu 16.04 |
| Scripting | Python 2.7 |
| Framework | Ansible 2.a.0.0 |
| OpenStack | Pike or Queens or Rocky |
2.3 Pre-requsites Requirements
- Machine running SNAPS-OpenStack should have Ubuntu 16.04 Xenial as host OS and should have internet access.
- All host machines should have identical interface names and should have at least 2 interfaces (one for management and one for data).
- All host machines are connected to Build Server (machine running SNAPS-OpenStack) and have Internet access connectivity via data interface.
- For DPDK data interface should be a DPDK capable NIC.
- For creating a DPDK enabled VM, user is required to configure flavor property "hw:mem_page_size" to "large"
Note: Build Server should have http/https and ftp proxy if node is behind corporate firewall. Set the http/https proxy for apt.
3 Configuration
3.1 deployment.yaml
Configuration file used by SNAPS-OpenStack for OpenStack provisioning. Options defined here are used by SNAPS-OpenStack to deploy appropriate OpenStack services on host machines and configuring them to be controller, compute or storage nodes.
OpenStack:
The trunk is currently Queens release. If you want to deploy Pike release, please checkout stable/pike branch of SNAPS-OpenStack.
versioning:
This section is used for OpenStack release planning. Parameters defined in this section allow user to specify OpenStack version. Configuration parameters defined in this section are explained below.
| Parameter | Optionality | Description |
|---|---|---|
| release | N | Name of the OpenStack release.(Use rocky for branch , or rocky\7.0.1 for tag based kolla deployment). |
| image | Y | images can be built or pulled from docker hub for deployment by setting build or pull (default is pull). |
| repo | Y | container images to be pulled from a specific docker hub namespace or the kolla namespace (default). |
| repo_tag | Y | tag name of the specific container images to be pulled for deployment. |
hosts:
This section is used for OpenStack environment planning. Parameters defined here configure host machines as OpenStack controller, compute, storage and network node. Configuration parameters defined in this section are explained below.
| Parameter | Optionality | Description | ||
|---|---|---|---|---|
| host | Define this set of parameters for each host machine (a separate host section should be defined for each host machine). | |||
| hostname | N | Hostname to be used for the machine. SNAPS-OpenStack assigns this hostname to the machine. | ||
| interfaces | Define this set of parameters for each interface of the host machine. | |||
| ip | N | IP of the primary interface (Management Interface, allocated after OS provisioning). | ||
| name | N | Name of the primary interface. | ||
| type | N | Traffic type (management). |
||
| gateway | N | Gateway Ip of the subnet. | ||
| ip | N | IP of the 2nd interface (External Network). | ||
| name | N | Name of 2nd interface. | ||
| type | N | Traffic type (data). |
||
| ip | Y | IP of 3rd interface (Tenant Network). | ||
| name | Y | Name of the interface. | ||
| type | Y | Traffic type (tenant). |
||
| node_type | N | List of nodes to be setup on this host machine. User can choose any combination of controller, compute, network and storage values. If user wishes to deploy a single node setup, he should use value all. |
||
| second_storage | Y | List of mount point of secondary storage for ceph. Has to be present if the node_type is "storage" | ||
| sriov_interface | Y | List of SRIOV interfaces for SRIOV. Has to be present if SRIOV is enabled | ||
| user | N | User of host machine. | ||
| password | N | Password for host machine user. | ||
| isolcpus | Y | CPUs to be pinned for VMs on this host machine. | ||
| reserved_host_memory_mb | Y | RAM to be reserved for VMs on this host machine. | ||
networks
SNAPS-OpenStack uses this section to define external and tenant networks for OpenStack VMs. This section is optional, if user does not provide this section, SNAPS-OpenStack will create default network. Configuration parameters defined in this section are explained below.
external
| Parameter | Optionality | Description | |
|---|---|---|---|
| gateway | N | Gateway IP for external network. | |
| ip_pool | |||
| End | N | Last address for DHCP range. | |
| Start | N | First address for DHCP range. | |
| Subnet | N | Subnet in CIDR notation. | |
tenant
| Parameter | Optionality | Description |
|---|---|---|
| Subnet | N | Subnet in CIDR notation. |
| subnet_size | N | Size of subnet. |
mtu_size
| Parameter | Optionality | Description |
|---|---|---|
| default | N | Default MTU size to be used on provider networks. |
| vxlan | N | MTU size to be used on overlay networks. |
Note: Default mtu size value should be greater than or equal to vxlan mtu size.
proxies
This section defines environment proxies to be exported on all host machines. Configuration parameters defined in this section are explained below.
| Parameter | Optionality | Description |
|---|---|---|
| ftp_proxy | Y | Proxy to be used for FTP. |
| http_proxy | Y | Proxy to be used for HTTP traffic. |
| https_proxy | Y | Proxy to be used for HTTPS traffic. |
| no_proxy | N | Comma separated list of IPs of all host machines. Localhost 127.0.0.1 should be included here. |
Note: use proxy only when direct access to internet not available.
service_password
Password to be used for OpenStack service endpoints.
services
Defines the additional services to be installed. Allowed values are magnum,
telemetry, cinder etc. See sample below:
services:
- magnum
- tempest
- ceilometer
- cinder
- tacker
- ceph
- sriov
- dpdk
- designate
kolla
This section is required only for Kolla based OpenStack deployment.
| Parameter | Optionality | Description |
|---|---|---|
| base_distro | N | Should be ubuntu. |
| install_type | N | Should be source. |
| keepalived_virtual_router_id | N | Should be a unique number for each deployment within a network domain. |
| internal_vip_address | N | Any unused IP address in the management network. |
| kolla_registry | N | IP of node where docker registry has to be created. In current version it should be controller node IP. |
| kolla_registry_port | N | Port to be used for docker registry. |
| internal_interface | N | Interface for the OpenStack internal api end points. |
| external_vip_address | N | Ip address for OpenStack external end points. |
| external_interface | N | Interface for the OpenStack external api end points. |
| base_size | Y | Base size for the physical volume of the cinder. |
| count | Y | Total count for the physical volume created. |
3.2 var.yaml (VLAN Configuration)
Configuration file used by SNAPS-OpenStack for VLAN based tenant network provisioning.
TASKS
Parameters defined in this section allow user to specify post deployment tasks (VLAN configuration). Configuration parameters defined in this section are explained below:
| Parameter | Optionality | Description | ||
|---|---|---|---|---|
| name | N | Should be TenantVLAN. |
||
| physical_network | N | Should be name of physical network to be used for VLAN. |
||
| min_vlan_id | N | Should be minimum value of vlan id to be used. |
||
| max_vlan_id | N | Should be maximum value of vlan id to be used. |
||
| host | Define this set of parameters for each host machine (A separate host section should be defined for each host machine). | |||
| interfaces | Define this set of parameters for each VLAN. | |||
| port_name | N | Interface name attached to the vlan. | ||
| size | N | Vlan MTU size. | ||
| ip | Y | IP of Management network. | ||
| username | N | User of host machine. | ||
| password | N | Password for host machine user. | ||
3.3 var.yaml (MTU settings)
Configuration file used by SNAPS-OpenStack for MTU size configuration of physical NICs.
TASKS
Parameters defined in this section allows user to specify post deployment tasks (MTU settings). Configuration parameters defined in this section are explained below.
| Parameter | Optionality | Description | ||
|---|---|---|---|---|
| name | N | Should be mtu. |
||
| host | Define this set of parameters for each host machine (A separate host section should be defined for each host machine). | |||
| interfaces | Define this set of parameters for each interface to be reconfigured for MTU size. | |||
| port_name | N | Interface name attached to the vlan. | ||
| size | N | MTU size. | ||
| ip | Y | IP of Management network. | ||
| username | N | User of host machine. | ||
| password | N | Password for host machine user. | ||
3.4 var.yaml (Single NIC settings)
Configuration file used by SNAPS-OpenStack for single NIC deployment.
TASKS
Parameters defined in this section allows user to specify pre and post deployment tasks (Single NIC). Configuration parameters defined in this section are explained below.
| Parameter | Optionality | Description | ||
|---|---|---|---|---|
| name | N | Should be pre-post. |
||
| host | Define this set of parameters for each host machine (A separate host section should be defined for each host machine). | |||
| ip | Y | IP of Management network. | ||
| node_type | N | User can choose node type controller/compute. | ||
| primary_interface | N | Name of primary interfaces. | ||
| target_interface | N | Name of the veth pairs to be created. | ||
4 Installation Steps
4.1 Fresh OpenStack Installation
Step 1
Clone/FTP SNAPS-OpenStack on to the Build Server via:
sudo git clone https://github.com/cablelabs/snaps-openstack.git
All operations on Build Server expect the user should be explicitly switched (using
su root) to the root user, or prepend 'sudo' to each command.
Step 2
Go to directory <repo_dir>/conf/openstack/kolla
Modify file deployment.yaml for provisioning of OpenStack nodes on cloud
cluster host machines (controller node, compute nodes). Modify this file
according to your set up environment only (Refer section 3).
Step 3
Run iaas_launch.py as shown below:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -drs
This will install Kolla OpenStack service on host machines. Your OpenStack installation will start and will get completed in about an hour.
Note: Since the deployment takes long, you probably want to run it within a screen session.
4.2 VLAN Tenant Network Configuration
Step 1
Go to <repo_dir>/snaps_openstack/utilities/ directory.
Define VLAN ports per host in var.yaml file under TenantVLAN task (we can
define multiple vlan ports for multiple hosts).
Step 2
Run network_config.py as shown below:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -tvlan
Please configure the switch for tagged vlan ports.
4.3 MTU Size Settings for Physical NICs
Step 1
Go to <repo_dir>/snaps_openstack/utilities/ directory.
Define MTU size for NICS per host in var.yaml file under mtu task.
Step 2
Run network_config.py as shown below:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -mtu
4.4 Settings for Single NIC deployment
Execute the following 4 steps only if deploying snaps-openstack to nodes with a single NIC .
Step 1
Go to <repo_dir>/snaps_openstack/utilities/ directory.
Define Single NIC configurations for all hosts in var.yaml file under pre-post task.
Step 2
Run network_config.py as shown below:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -preNic
Step 3
Install fresh openstack using physical interface and veth0 as management and data interfaces respectively. (Refer to section 4)
Step 4
Run network_config.py as shown below:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -postNic
5 Upgrade and Downgrade Openstack Cluster
5.1 Upgrade Openstack Cluster
After successful installation of openstack cluster with Pike release (Refer to section 4). User can upgrade the setup to Queens release by running the following command:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -upgrade queens
5.2 Downgrade Openstack Cluster
After successful installation of openstack cluster with Queens release (Refer to section 4). User can downgrade the setup to Pike release by running the following command:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -downgrade pike
6 Cleanup and Troubleshooting
6.1 Fresh Deployment with Existing Docker Repository
If docker based images for OpenStack services are already available run
iaas_launch.py as shown below:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -d
6.2 Re-deployment
In case previous deployment attempt has failed or new changes are required (enabling optional services), attempt following steps.
First, Clean up previous OpenStack deployment:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -c
Or Clean up previous deployment along with docker repository:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -drc
Note: This step will reboot each target server when it is done. Wait a few minutes then ping and/or ssh each target server to verify it is back up. In addition, log into each target server to verify that docker processes and images are cleaned up. If not, clean up the docker processes and images manually to prevent them from causing re-deployment failures.
Last, re-install OpenStack. If docker repository exists:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -d
Or if docker repository needs to be built:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -drs
6.3 Cleanup
Clean up previous OpenStack deployment:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -c
Clean up previous deployment along with docker repository:
sudo python <repo_dir>/iaas_launch.py -f <repo_dir>/conf/openstack/kolla/deployment.yaml -drc
Clean up previous vlan configuration:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -tvclean
6.3.1 Vlan Mapping Cleanup
Only perform the steps below after you've run vlan configuration cleanup script as instructed in 6.3 above. The manual cleanup steps in this section are necessary to workaround an upstream defect in https://bugs.launchpad.net/neutron/+bug/1743425:
Ssh to control node:
sudo ssh <control-node-ip>
Obtain the maria db root credential:
grep wsrep_sst_auth /etc/kolla/mariadb/galera.cnf | cut -d":" -f2
Run interactive bash shell from mariadb container:
docker exec -ti mariadb bash
Connect to neutron DB as root (enter maria db root credential when prompted):
mysql neutron -u root -p
Remove vlan mappings from neutron DB:
TRUNCATE TABLE ml2_vlan_allocations;
Exit from neutron DB:
exit
Exit from mariadb container interactive mode:
exit
6.3.2 Single NIC Cleanup
Before cleaning the opesntack setup user needs to clean up the single NIC configurations.
Run network_config.py as shown below:
sudo python <repo_dir>/network_config.py -f <repo_dir>/snaps_openstack/utilities/var.yaml -cleanNic