README.md 4.56 KB
Newer Older
Jean-Francois Rey's avatar
Jean-Francois Rey committed
1
2
3
4
5
6
[![pipeline status](https://gitlab.paca.inrae.fr/r-ecosystem/windowsr/badges/master/pipeline.svg)](https://gitlab.paca.inrae.fr/r-ecosystem/windowsr/-/commits/master)


Original repository : [https://gitlab.paca.inrae.fr/r-ecosystem/windowsr/](https://gitlab.paca.inrae.fr/r-ecosystem/windowsr/)  


Jean-Francois Rey's avatar
Jean-Francois Rey committed
7
# Windows 10 Virtual Machine, R environment, as GitLab Runner
Jean-Francois Rey's avatar
Jean-Francois Rey committed
8

Jean-Francois Rey's avatar
Jean-Francois Rey committed
9
If you want to develop, tests and check your R packages, build R packages binaries under/for Windows automatically, this is the right place.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
10

Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
11
> __Create a VirtualBox under windows 10 with R (Version >= 4.0.0) and Rtools + some dependencies.  And register it as a GitLab Runner.__  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
12
13

[[_TOC_]]
Jean-Francois Rey's avatar
Jean-Francois Rey committed
14

Jean-Francois Rey's avatar
Jean-Francois Rey committed
15
16
17
18
19
20
## The Idea

![Project pipeline windows R](images/GitLab_Runner_pipeline_R.png)

__The main purpose of this project is to automate the creation of a R environment under Windows to use it as a GitLab Runner in CI/CD pipelines.__  

21
Here is the steps :  
Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
22
23
24
1. __Get__ a VirtualBox Machine from [VagrantCloud](https://app.vagrantup.com/) (a box) under windows 10, generated by [Packer](https://www.packer.io/). More information [here](https://forgemia.inra.fr/gaev/packer/windows10).  
2. __Initializing__ and configuring the VM using [Vagrant](https://www.vagrantup.com/). See [VagrantFile](VagrantFile) template.  
3. __Provisioning__ and configuring the VM using [Ansible](https://ww.ansible.com). See [ansible\_provisioning/playbook.yml](ansible\_provisioning/playbook.yml) and scripts in [ansible\_provisioning/files/](ansible\_provisioning/files/) :   
Jean-Francois Rey's avatar
Jean-Francois Rey committed
25
26
27
28
29
30
31
32
33
	- Update Windows
	- Install R (>= 4.0.0)  
	- Install Rtools (>= Rtools40)  
	- Configure msys2 environment (Rtools dependencies)  
	- Install sshd (under msys2)  
	- Install some R packages and system dependencies  
	- Install GitLab-Runner for artifacts (lastest version)  
	- Install MiKTex (lastest version)  
	- Install Pandoc (2.10.1)  
Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
34
35
4. __Generating__ a new Vagrant box  
5. __Registering__ this new VM as a GitLab Runner (snapshot version)  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
36

Jean-Francois Rey's avatar
Jean-Francois Rey committed
37
38
## Dependencies

Jean-Francois Rey's avatar
Jean-Francois Rey committed
39
A bare metal or a Virtual Machine with nested Virtualization on the hypervisor activated as a server with dependencies installed. (Tested under Linux OS)    
40

Jean-Francois Rey's avatar
Jean-Francois Rey committed
41
### System
Jean-Francois Rey's avatar
Jean-Francois Rey committed
42

Jean-Francois Rey's avatar
Jean-Francois Rey committed
43
44
45
46
47
* [VirtualBox](https://www.virtualbox.org/) (>=6.1.14) and extension Pack
* [Vagrant](https://www.vagrantup.com/) (>=2.2.10)
* [Ansible](https://www.ansible.com/) (>=2.2.1.0)
* [pywinrm](https://github.com/diyan/pywinrm) (>=0.3.0) compatible with Ansible Python version
* [GitLab-Runner](https://docs.gitlab.com/runner/install/) (>= 10.0.0)
Jean-Francois Rey's avatar
Jean-Francois Rey committed
48
49
50

### GitLab Runner

Jean-Francois Rey's avatar
Jean-Francois Rey committed
51
52
Use GitLab-Runner to register a _shell_ runner. _gitlab-runner_ user have to get access to _vboxmanage_, _vagrant_ and _ansible_.  

Jean-Francois Rey's avatar
Jean-Francois Rey committed
53
To register the VirtualBox Runner created here, called _WIN10R4_.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
54
55
56
57
58
59
60
61
62
63
64
65
66
67
```bash
gitlab-runner register \
	--non-interactive \
	--name win10R4
	--url <Gitlab_URL> \
	--registration-token <registration-token> \
	--executor "virtualbox" \
	--tag-list windows,R4 \
	--ssh-user vagrant \
  --ssh-password vagrant \
	--virtualbox-base-name "WIN10R4" \
	--virtualbox-disable-snapshots = false \
	--virtualbox-base-snapshot = "bootR4"
```
68

Jean-Francois Rey's avatar
Jean-Francois Rey committed
69
To use this Runner, add in your .gitlab-ci.yml jobs:
Jean-Francois Rey's avatar
Jean-Francois Rey committed
70
71
72
73
74
```
tags:
  - windows
  - R4
```
Jean-Francois Rey's avatar
Jean-Francois Rey committed
75

Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
76
## Principle
Jean-Francois Rey's avatar
Jean-Francois Rey committed
77
	
Jean-Francois Rey's avatar
Jean-Francois Rey committed
78
Every things happen in the [.gitlab-ci.xml](.gitlab-ci.xml) file.  
Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
79

Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
80
Mainly the [VagrantFile](VagrantFile) set up a virtual machine from __Vagrant Cloud__ [GAEV/win10](https://app.vagrantup.com/GAEV/boxes/win10) box (a windows10 professional evaluation version with vbBoxGuestAditions).  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
81
Files in [ansible\_provisioning/](ansible_provisioning/) are playbooks for __Ansible__ that will install/configure R, Rtools, packages, MiKTeX, Pandoc and GitLab-Runner using 'roles'.  
Jean-Francois Rey's avatar
up doc    
Jean-Francois Rey committed
82
83
After the Virtual Machine creation and configuration, it registered as a new __gitlab-runner__ (by replacing the existing virtualbox machine "WIN10R4").  
Bonus : The VM (box) is save/backup in a __OpenStack Object Store__ [Swift](https://wiki.openstack.org/wiki/Swift).  
Jean-Francois Rey's avatar
update    
Jean-Francois Rey committed
84

Jean-Francois Rey's avatar
Jean-Francois Rey committed
85
86
87
88
89
90
91
92
93
94
95
## Locally

To set up the VM (called windows10R4) locally (on your computer), you just need to run :  
```bash
vagrant up
vagrant snapshot bootR4
vagrant halt
```

## Customization

Jean-Francois Rey's avatar
Jean-Francois Rey committed
96
97
* To add R packages : add _r\_pkg\_cran_ variable in playbook.yml or edit [ansible\_provisioning/roles/R-packages/default.yml](ansible\_provisioning/roles/R-packages/default.yml) and add systems libraries and R packages to install.  
* Make artifacts work on your GitLab instance : edit file [ansible\_provisioning/roles/cicd/files/install-gitlab-runner.sh](ansible\_provisioning/roles/cicd/files/install-gitlab-runner.sh) and change the gitlab URL and IP.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
98
* Custom VirtualBox : edit file [VagrantFile](VagrantFile) and modify parameters in virtualbox provider.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
99