README.md 4.58 KB
Newer Older
Jean-Francois Rey's avatar
Jean-Francois Rey committed
1
2
3
4
5
[![pipeline status](https://gitlab.paca.inrae.fr/r-ecosystem/macosr/badges/master/pipeline.svg)](https://gitlab.paca.inrae.fr/r-ecosystem/macosr/-/commits/master)  

Original repository : [https://gitlab.paca.inrae.fr/r-ecosystem/macosr](https://gitlab.paca.inrae.fr/r-ecosystem/macosr)  


Jean-Francois Rey's avatar
Jean-Francois Rey committed
6
# MacOS, R environment, as GitLab CI/CD
Jean-Francois Rey's avatar
Jean-Francois Rey committed
7

Jean-Francois Rey's avatar
Jean-Francois Rey committed
8
9
10
11
12
If you want to develop and test your R packages, build R packages binaries under/for MacOS automatically, this is the right place.

> __This project is only for developpement and testing.  For other use you must have Apple equipment.__


Jean-Francois Rey's avatar
Jean-Francois Rey committed
13
14
## What is doing ?

Jean-Francois Rey's avatar
Jean-Francois Rey committed
15
16
> Create a VirtualBox under MacOS Catalina 10.15 with R (>= 4.0.0) and Rtools + some dependencies. And register it as a GitLab-Runner.

Jean-Francois Rey's avatar
Jean-Francois Rey committed
17
18
19
20
21
22
[[_TOC_]]

## The Idea

![Project pipeline windows R](images/GitLab_Runner_pipeline_R.png)

Jean-Francois Rey's avatar
Jean-Francois Rey committed
23
__The main purpose of this project is to automate the creation of a R environment under MacOS to use it as a GtiLab-Runner in CI/CD pipelines.__  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

Here is the steps :  
1. __Get__ a VirtualBox Machine from [VagrantCloud](https://app.vagrantup.com/) (a box) under MacOS, generated by [Packer](https://www.packer.io/). More information [here](https://forgemia.inra.fr/gaev/packer/macos).  
2. __Initializing__ and configuring the VM using [Vagrant](https://www.vagrantup.com/). See [VagrantFile](VagrantFile) template.  
3. __Provisioning__ and configuring the VM using [Ansible](https://ww.ansible.com). See [ansible\_provisioning/playbook.yml](ansible\_provisioning/playbook.yml) and scripts in [ansible\_provisioning/files/](ansible\_provisioning/files/) :   
  - Update Mac OS (optional)  
  - Install R-4.0-branch (>= 4.0.0) / lastest stable version  
  - Install gfortran 10.2 [R tools chain](https://github.com/fxcoudert/gfortran-for-macOS/releases)  
	- Install static libs (R packages libs dependencies) [https://mac.r-project.org/libs-4/](https://mac.r-project.org/libs-4/)  
	- Install some R packages
  - Install GitLab-Runner for artifacts (lastest version)  
  - Install MacTex (lastest version)  
	- Install XQuartz (v2.7.11)  
4. __Generating__ a new Vagrant box  
5. __Registering__ this new VM as a GitLab Runner (snapshot version)  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
39
40
41
42
43

## Dependencies

A bare metal or a virtual machine (with nested virtualization on the hypervisor activated) as a server with dependencies installed. Apple equipement.

Jean-Francois Rey's avatar
Jean-Francois Rey committed
44
### System
Jean-Francois Rey's avatar
Jean-Francois Rey committed
45

Jean-Francois Rey's avatar
Jean-Francois Rey committed
46
47
48
49
50
* [VirtualBox](https://www.virtualbox.org/) (>= 6.1.14)
* [Vagrant](https://www.vagrantup.com/) (>= 2.2.10)
* [Ansible](https://www.ansible.com/) (>=2.2.1.0)
* sshpass (>= 1.06)
* [GitLab-Runner](https://docs.gitlab.com/runner/install/) (>= 10.0.0)
Jean-Francois Rey's avatar
Jean-Francois Rey committed
51
52
53
54
55
56
57
58
59

### GitLab Runner

Use GitLab-Runner to register a _shell_ runner. _gitlab-runner_ user have to get access to _vboxmanage_, _vagrant_ and _ansible_.  

To register the virtualbox runner create here, called _MACOSCR4_.  
```bash
gitlab-runner register \
  --non-interactive \
Jean-Francois Rey's avatar
Jean-Francois Rey committed
60
  --name MacOSCR4
Jean-Francois Rey's avatar
Jean-Francois Rey committed
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
  --url <Gitlab_URL> \
  --registration-token <registration-token> \
  --executor "virtualbox" \
  --tag-list MacOS,R4 \
  --ssh-user vagrant \
  --ssh-password vagrant \
  --virtualbox-base-name "MACOSCR4" \
  --virtualbox-disable-snapshots = false \
  --virtualbox-base-snapshot = "bootR4"
```

To use this runner, add in your .gitlab-ci.yml jobs:
```
tags:
  - MacOS
  - R4
```

## Principle

Every things happen in the [.gitlab-ci.xml](.gitlab-ci.xml) file.  

Mainly the [VagrantFile](VagrantFile) set up a virtual machine from Vagrant Cloud [GAEV/MacOS\_Catalina](https://app.vagrantup.com/GAEV/boxes/MacOS_Catalina) box.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
84
Files in [provisioning/](provisioning/) are playbooks for __Ansible__ that will install/configure R, tools [https://mac.r-project.org/](https://mac.r-project.org/), packages and dependencies, MacTeX, Xquartz and GitLab-Runner.  
Jean-Francois Rey's avatar
Jean-Francois Rey committed
85
After the Virtual Machine creation and configuration, it registered as a new __gitlab-runner__ (by replacing the existing virtualbox machine "MACOSCR4").
Jean-Francois Rey's avatar
Jean-Francois Rey committed
86
87
88
89
90
91
92
93
94
95
96
97

## Locally

To set up the VM (called _MacOSCR4_) locally (on your computer), you just need to run :  
```bash
vagrant up
vagrant snapshot bootR4
vagrant halt
```

## Customization

Jean-Francois Rey's avatar
Jean-Francois Rey committed
98
99
* To add R packages : edit r\_pkg\_cran variable or edit file [ansible\_provisioning/roles/R-packages/default/main.yml](ansible\_provisioning/roles/R-packages/default/main.yml) and add systems libraries [libs-4](https://mac.r-project.org/libs-4/) and R packages to install.  
* Make artifacts work on your GitLab instance : edit file [ansible\_provisioning/roles/cicd/files/install-gitlabrunner.sh](ansible\_provisioning/roles/cicd/files/install-gitlabrunner.sh) and change the gitlab URL and IP.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
100
* Custom VirtualBox : edit file [VagrantFile](VagrantFile) and modify parameters in virtualbox provider.
Jean-Francois Rey's avatar
Jean-Francois Rey committed
101