README.md 4.23 KB
Newer Older
Jean-Francois Rey's avatar
Jean-Francois Rey committed
1
# MacOS, R environment, as GitLab CI/CD
Jean-Francois Rey's avatar
Jean-Francois Rey committed
2

Jean-Francois Rey's avatar
Jean-Francois Rey committed
3
4
5
6
7
If you want to develop and test your R packages, build R packages binaries under/for MacOS automatically, this is the right place.

> __This project is only for developpement and testing.  For other use you must have Apple equipment.__


Jean-Francois Rey's avatar
Jean-Francois Rey committed
8
9
## What is doing ?

Jean-Francois Rey's avatar
Jean-Francois Rey committed
10
11
> Create a VirtualBox under MacOS Catalina 10.15 with R (>= 4.0.0) and Rtools + some dependencies. And register it as a GitLab-Runner.

Jean-Francois Rey's avatar
Jean-Francois Rey committed
12
13
14
15
16
17
[[_TOC_]]

## The Idea

![Project pipeline windows R](images/GitLab_Runner_pipeline_R.png)

Jean-Francois Rey's avatar
Jean-Francois Rey committed
18
__The main purpose of this project is to automate the creation of a R environment under MacOS to use it as a GtiLab-Runner in CI/CD pipelines.__  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

Here is the steps :  
1. __Get__ a VirtualBox Machine from [VagrantCloud](https://app.vagrantup.com/) (a box) under MacOS, generated by [Packer](https://www.packer.io/). More information [here](https://forgemia.inra.fr/gaev/packer/macos).  
2. __Initializing__ and configuring the VM using [Vagrant](https://www.vagrantup.com/). See [VagrantFile](VagrantFile) template.  
3. __Provisioning__ and configuring the VM using [Ansible](https://ww.ansible.com). See [ansible\_provisioning/playbook.yml](ansible\_provisioning/playbook.yml) and scripts in [ansible\_provisioning/files/](ansible\_provisioning/files/) :   
  - Update Mac OS (optional)  
  - Install R-4.0-branch (>= 4.0.0) / lastest stable version  
  - Install gfortran 10.2 [R tools chain](https://github.com/fxcoudert/gfortran-for-macOS/releases)  
	- Install static libs (R packages libs dependencies) [https://mac.r-project.org/libs-4/](https://mac.r-project.org/libs-4/)  
	- Install some R packages
  - Install GitLab-Runner for artifacts (lastest version)  
  - Install MacTex (lastest version)  
	- Install XQuartz (v2.7.11)  
4. __Generating__ a new Vagrant box  
5. __Registering__ this new VM as a GitLab Runner (snapshot version)  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
34
35
36
37
38

## Dependencies

A bare metal or a virtual machine (with nested virtualization on the hypervisor activated) as a server with dependencies installed. Apple equipement.

Jean-Francois Rey's avatar
Jean-Francois Rey committed
39
### System
Jean-Francois Rey's avatar
Jean-Francois Rey committed
40

Jean-Francois Rey's avatar
Jean-Francois Rey committed
41
42
43
44
45
* [VirtualBox](https://www.virtualbox.org/) (>= 6.1.14)
* [Vagrant](https://www.vagrantup.com/) (>= 2.2.10)
* [Ansible](https://www.ansible.com/) (>=2.2.1.0)
* sshpass (>= 1.06)
* [GitLab-Runner](https://docs.gitlab.com/runner/install/) (>= 10.0.0)
Jean-Francois Rey's avatar
Jean-Francois Rey committed
46
47
48
49
50
51
52
53
54

### GitLab Runner

Use GitLab-Runner to register a _shell_ runner. _gitlab-runner_ user have to get access to _vboxmanage_, _vagrant_ and _ansible_.  

To register the virtualbox runner create here, called _MACOSCR4_.  
```bash
gitlab-runner register \
  --non-interactive \
Jean-Francois Rey's avatar
Jean-Francois Rey committed
55
  --name MacOSCR4
Jean-Francois Rey's avatar
Jean-Francois Rey committed
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
  --url <Gitlab_URL> \
  --registration-token <registration-token> \
  --executor "virtualbox" \
  --tag-list MacOS,R4 \
  --ssh-user vagrant \
  --ssh-password vagrant \
  --virtualbox-base-name "MACOSCR4" \
  --virtualbox-disable-snapshots = false \
  --virtualbox-base-snapshot = "bootR4"
```

To use this runner, add in your .gitlab-ci.yml jobs:
```
tags:
  - MacOS
  - R4
```

## Principle

Every things happen in the [.gitlab-ci.xml](.gitlab-ci.xml) file.  

Mainly the [VagrantFile](VagrantFile) set up a virtual machine from Vagrant Cloud [GAEV/MacOS\_Catalina](https://app.vagrantup.com/GAEV/boxes/MacOS_Catalina) box.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
79
Files in [provisioning/](provisioning/) are playbooks for __Ansible__ that will install/configure R, tools [https://mac.r-project.org/](https://mac.r-project.org/), packages and dependencies, MacTeX, Xquartz and GitLab-Runner.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
80
After the Virtual Machine creation and configuration, it registered as a new __gitlab-runner__ (by replacing the existing virtualbox machine "MACOSCR4").
Jean-Francois Rey's avatar
Jean-Francois Rey committed
81
82
83
84
85
86
87
88
89
90
91
92

## Locally

To set up the VM (called _MacOSCR4_) locally (on your computer), you just need to run :  
```bash
vagrant up
vagrant snapshot bootR4
vagrant halt
```

## Customization

Jean-Francois Rey's avatar
Jean-Francois Rey committed
93
* To add R packages : edit file [ansible\_provisioning/files/install-r-library.sh](ansible\_provisioning/files/install-r-library.sh) and add systems libraries [libs-4](https://mac.r-project.org/libs-4/) and R packages to install.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
94
95
* Make artifacts work on your GitLab instance : edit file [ansible\_provisioning/files/install-gitlabrunner.sh](ansible\_provisioning/files/install-gitlabrunner.sh) and change the gitlab URL and IP.
* Custom VirtualBox : edit file [VagrantFile](VagrantFile) and modify parameters in virtualbox provider.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
96