README.md 4.19 KB
Newer Older
Jean-Francois Rey's avatar
Jean-Francois Rey committed
1
# Windows 10 Virtual Machine, R environment, as GitLab Runner
Jean-Francois Rey's avatar
Jean-Francois Rey committed
2

Jean-Francois Rey's avatar
Jean-Francois Rey committed
3
If you want to develop, tests and check your R packages, build R packages binaries under/for Windows automatically, this is the right place.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
4

Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
5
> __Create a VirtualBox under windows 10 with R (Version >= 4.0.0) and Rtools + some dependencies.  And register it as a GitLab Runner.__  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
6
7

[[_TOC_]]
Jean-Francois Rey's avatar
Jean-Francois Rey committed
8

Jean-Francois Rey's avatar
Jean-Francois Rey committed
9
10
11
12
13
14
15
## The Idea

![Project pipeline windows R](images/GitLab_Runner_pipeline_R.png)

__The main purpose of this project is to automate the creation of a R environment under Windows to use it as a GitLab Runner in CI/CD pipelines.__  

Here the steps :  
Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
16
17
18
1. __Get__ a VirtualBox Machine from [VagrantCloud](https://app.vagrantup.com/) (a box) under windows 10, generated by [Packer](https://www.packer.io/). More information [here](https://forgemia.inra.fr/gaev/packer/windows10).  
2. __Initializing__ and configuring the VM using [Vagrant](https://www.vagrantup.com/). See [VagrantFile](VagrantFile) template.  
3. __Provisioning__ and configuring the VM using [Ansible](https://ww.ansible.com). See [ansible\_provisioning/playbook.yml](ansible\_provisioning/playbook.yml) and scripts in [ansible\_provisioning/files/](ansible\_provisioning/files/) :   
Jean-Francois Rey's avatar
Jean-Francois Rey committed
19
20
21
22
23
24
25
26
27
	- Update Windows
	- Install R (>= 4.0.0)  
	- Install Rtools (>= Rtools40)  
	- Configure msys2 environment (Rtools dependencies)  
	- Install sshd (under msys2)  
	- Install some R packages and system dependencies  
	- Install GitLab-Runner for artifacts (lastest version)  
	- Install MiKTex (lastest version)  
	- Install Pandoc (2.10.1)  
Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
28
29
4. __Generating__ a new Vagrant box  
5. __Registering__ this new VM as a GitLab Runner (snapshot version)  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
30

Jean-Francois Rey's avatar
Jean-Francois Rey committed
31
32
## Dependencies

Jean-Francois Rey's avatar
Jean-Francois Rey committed
33
A bare metal or a Virtual Machine with nested Virtualization on the hypervisor activated as a server with dependencies installed. (Tested under Linux OS)    
34

Jean-Francois Rey's avatar
Jean-Francois Rey committed
35
### System
Jean-Francois Rey's avatar
Jean-Francois Rey committed
36

Jean-Francois Rey's avatar
Jean-Francois Rey committed
37
38
39
40
41
* [VirtualBox](https://www.virtualbox.org/) (>=6.1.14) and extension Pack
* [Vagrant](https://www.vagrantup.com/) (>=2.2.10)
* [Ansible](https://www.ansible.com/) (>=2.2.1.0)
* [pywinrm](https://github.com/diyan/pywinrm) (>=0.3.0) compatible with Ansible Python version
* [GitLab-Runner](https://docs.gitlab.com/runner/install/) (>= 10.0.0)
Jean-Francois Rey's avatar
Jean-Francois Rey committed
42
43
44

### GitLab Runner

Jean-Francois Rey's avatar
Jean-Francois Rey committed
45
46
Use GitLab-Runner to register a _shell_ runner. _gitlab-runner_ user have to get access to _vboxmanage_, _vagrant_ and _ansible_.  

Jean-Francois Rey's avatar
Jean-Francois Rey committed
47
To register the VirtualBox Runner created here, called _WIN10R4_.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
48
49
50
51
52
53
54
55
56
57
58
59
60
61
```bash
gitlab-runner register \
	--non-interactive \
	--name win10R4
	--url <Gitlab_URL> \
	--registration-token <registration-token> \
	--executor "virtualbox" \
	--tag-list windows,R4 \
	--ssh-user vagrant \
  --ssh-password vagrant \
	--virtualbox-base-name "WIN10R4" \
	--virtualbox-disable-snapshots = false \
	--virtualbox-base-snapshot = "bootR4"
```
62

Jean-Francois Rey's avatar
Jean-Francois Rey committed
63
To use this Runner, add in your .gitlab-ci.yml jobs:
Jean-Francois Rey's avatar
Jean-Francois Rey committed
64
65
66
67
68
```
tags:
  - windows
  - R4
```
Jean-Francois Rey's avatar
Jean-Francois Rey committed
69

Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
70
## Principle
Jean-Francois Rey's avatar
Jean-Francois Rey committed
71
	
Jean-Francois Rey's avatar
Jean-Francois Rey committed
72
Every things happen in the [.gitlab-ci.xml](.gitlab-ci.xml) file.  
Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
73

Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
74
75
76
77
Mainly the [VagrantFile](VagrantFile) set up a virtual machine from __Vagrant Cloud__ [GAEV/win10](https://app.vagrantup.com/GAEV/boxes/win10) box (a windows10 professional evaluation version with vbBoxGuestAditions).  
Files in [ansible_provisioning/](ansible_provisioning/) are playbooks for __Ansible__ that will install/configure R, Rtools, packages, MiKTeX, Pandoc and GitLab-Runner.  
After the Virtual Machine creation and configuration, it registered as a new __gitlab-runner__ (by replacing the existing virtualbox machine "WIN10R4").  
Bonus : The VM (box) is save/backup in a __OpenStack Object Store__ [Swift](https://wiki.openstack.org/wiki/Swift).  
Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
78

Jean-Francois Rey's avatar
Jean-Francois Rey committed
79
80
81
82
83
84
85
86
87
88
89
90
91
92
## Locally

To set up the VM (called windows10R4) locally (on your computer), you just need to run :  
```bash
vagrant up
vagrant snapshot bootR4
vagrant halt
```

## Customization

* To add R packages : edit file [ansible\_provisioning/files/install-r-packages.sh](ansible\_provisioning/files/install-r-packages.sh) and add systems libraries and R packages to install.  
* Make artifacts work on your GitLab instance : edit file [ansible\_provisioning/files/install-gitlab-runner.sh](ansible\_provisioning/files/install-gitlab-runner.sh) and change the gitlab URL and IP.
* Custom VirtualBox : edit file [VagrantFile](VagrantFile) and modify parameters in virtualbox provider.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
93