Lisa Anders - Posit Solutions Engineer and self-professed R nerd learning to love Python. Engineer turned data scientist turned Posit admin and excited to share lessons learned the hard way to make things easier for others. Also awkward at writing intros in the third person!
“Illustration from Hadley Wickham’s talk”The Joy of Functional Programming (for Data Science).” by Allison Horst
It’s Dangerous to go Alone! Take This
Imagine a world where code doesn’t disappear and is backed up in version control, environments are explicit and easily reproduced, and secrets are responsibly managed.
Who here has heard of git? Is anyone already using version control?
Version control: What is this thing called “git”?
Git was developed by the one and only Linus Torvalds (of linux fame), initially released in 2005 (19 years ago!)
Git itself is an open source utility, but most commonly it is used through a managed service (which are all pretty much the same under the hood). Here is a more complete list of version control software options:
You can check that SSH keys exist and the name with:
ls ~/.ssh
ssh-keygen -p
Version control: Credentials (SSH)
From RStudio:
Go to Tools -> Global Options -> Git / SVN
Create SSH Key
Approve the key and add a password (if appropriate)
View Public Key
Copy that public key that was provided into the SSH and GPG keys section in your git under your profile settings.
You’ll copy / update code using the SSH method from git.
Can also be done directly in the terminal
Version control: Oh no things have gone off the rails!
Troubleshooting from R:
library(usethis)library(gitcreds)library(gh)library(credentials)usethis::gh_token_help()usethis::git_sitrep()gh::gh_whoami()# Remove any pre-existing credentials:gitcreds::gitcreds_delete()
Troubleshooting from command line:
# Check the git configuration:git config --list# Check if an existing repository was cloned with ssh or https:git remote show origin# You can check that SSH keys exist and the name with:ls ~/.sshssh-keygen-p
Version control: Oh no things have gone off the rails!
Troubleshooting git when working in cloud environments:
On a cloud instance check that is is writing to somewhere persistent, IE into /home/ and not into /opt/. Find where your library is:
Centralized workflow: Teams use only a single repository and commit directly to the main branch.
Feature branching: Teams use a new branch for each feature and don’t commit directly to the main branch.
GitFlow: An extreme version of feature branching in which development occurs on the develop branch, moves to a release branch, and merges into the main branch.
Personal branching: Similar to feature branching, but rather than develop on a branch per feature, it’s per developer. Every user merges to the main branch when they complete their work.
Write meaningful commit messages and commits (“Made ABC do 123”)
Make small changes and merge often
Feedback from reviews is invaluable
As a reviewer:
Clearly distinguish between required changes, suggested changes, and just good ol’ recommendations or thoughts. It’s easy for reviews to come across as micro-managing, and this can help in distinguishing between what changes are needed vs what is just knowledge sharing.
Environment management example: Using renv, git, and Public Package Manager
Step 2: Use environment tracking
# Set up a new version controlled R project and install renv:install.packages("renv")library(renv)# Initialize your project with renv and take a snapshot:renv::init()renv::snapshot():# Update all packages, or revert back to an earlier snapshot:renv::update()renv::revert()# History is saved into version control:renv::history()
Environment management example: Using renv, git, and Public Package Manager
Step 3: Easy collaboration
# Have your colleague configure their repository to match yours: options(repos =c(REPO_NAME ="https://packagemanager.rstudio.com/all/latest")) ## Send a colleague the link to your project on git, they'll restore your environment with:renv::restore()
Environment management example: Using venv, git, and Public Package Manager
Follow the same steps, as with R
Step 1: Use pre-compiled packages
# Configure pip to use packages from public package manager: pip config set global.index-url https://packagemanager.posit.co/pypi/latest/simplepip config set global.trusted-host packagemanager.posit.co
Step 2: Use environment tracking
# Activate your virtual environment: python-m venv .venv. .venv/bin/activate# Take a snapshot of the environment: pip freeze > requirements.txt
Step 3: Easy collaboration
# Send a colleague the link to your project on git, they'll restore your environment with:pip install -r requirements.txt
Prevent exposing secret keys, passwords, sensitive config parameters in your code directly. This keeps those pieces more secure and less likely to escape into the wild.
usethis has a function for creating and editing the .Renviron file:
library(usethis)# Edit the global .Renviron fileusethis::edit_r_environ()# Edit the project specific .Renviron fileusethis::edit_r_environ(scope ="project")
Add the variables to that file in the format variable_name = "variable_value" and save it. Restart the session so the new environment variables will be loaded with ctrl shift f10 or through the RStudio IDE with session -> restart R.
Saved variables can be accessed with:
variable_name <-Sys.getenv("variable_name")
Secrets and Python
Same as with R, environment variables can be referenced and used in your code:
import os# Setting a new environment variableos.environ["API_KEY"] ="YOUR_API_KEY"# Retrieving the environment variablevar = os.environ["variable_name"]
Secrets and git
The .gitignore file is powerful! Tell git to ignore certain files that exist locally.
While typically explicitly listing the file name is the desired addition, wildcards can be added to exclude a type of file. For example: *.html.
Here’s an example .gitignore:
# History files
.Rhistory
.Rapp.history
# Session Data files
.RData
# Example code in package build process
*-Ex.R
# Output files from R CMD build
/*.tar.gz
# Output files from R CMD check
/*.Rcheck/
# RStudio files
.Rproj.user/
# produced vignettes
vignettes/*.html
vignettes/*.pdf
# OAuth2 token, see https://github.com/hadley/httr/releases/tag/v0.3
.httr-oauth
# knitr and R markdown default cache directories
/*_cache/
/cache/
# Temporary files created by R markdown
*.utf8.md
*.knit.md
# Shiny token, see https://shiny.rstudio.com/articles/shinyapps.html
rsconnect/
# Deployment details from rsconnect-python
rsconnect-python/
# Temporary files
.DS_Store
__pycache__
.ipynb_checkpoints
rmarkdown-notebook/flights.csv
.venv
venv
.env
.Rprofile
/.luarc.json
/.quarto/
Secrets and Posit Connect
Starting with version 1.6, RStudio Connect allows Environment Variables. The variables are encrypted on-disk, and in-memory.
Adding a variable through the Posit Connect UI is easy peasy:
Imagine a world where code doesn’t disappear and is backed up in version control, environments are explicit and easily reproduced, and secrets are responsibly managed. Wouldn’t that be great?
