Limit CPU/Memory When using Docker

After having trouble with the Singularity Pull command on a computing cluster, I decided it might be easier to download the image via Docker to my home machine and upload it from there. I tried it, but Docker turned out to be such a huge resource hog that it rendered my computer unusable. Okay, the thing to do is to limit the resources Docker is allowed; but a quick search seemed to indicate that, while plenty of people had the problem, the only thing to do was to not use virtual machines. Now that is great advice and I wholeheartedly endorse it; but sometimes it's inevitable. Eventually I got to an obscure SuperUser post that had an answer with zero upvotes that pointed to a blog post that explained what to do:

https://itnext.io/wsl2-tips-limit-cpu-memory-when-using-docker-c022535faf6f

Summary: Limit the CPU's and memory available to WSL2 (Docker's underlying mechanism) via a config file. I'll want to remove those limitations pretty quickly as I use WSL pretty heavily, but until I get these few tasks completed at least this will allow me to keep using my machine.

Including config.h in every file in Visual Studio

Rather than explicitly including a header file in every source file, I like to use gcc's -include flag to add a config.h file, a file holding global configuration options that every source file may or may not be using. For a long time I didn't think that Visual Studio had a similar option, but I finally dug it out here:

https://docs.microsoft.com/en-us/cpp/build/reference/fi-name-forced-include-file?view=msvc-160

The flag is /FI and you can set it in Visual Studio by right-clicking the project, selecting Properties, then C/C++, Advanced, and setting the "Forced Include File" property. Convenient!

Julia on a multi-user system

Had occasion to install Julia on a multi-user system today. I downloaded the tarball to my own directory and ran Make. The instructions say that the install is fully contained in the single directory, so you don't have to worry about files being installed in different locations on the system. Once it finished, I moved the directory to a globally accessible location and tried it out. It worked mostly, but nothing about the package

manager would run properly. Eventually I realized that if the directory that I had initially created existed, the package manager worked, but if I deleted it, the package manager stopped working. 

I deleted everything, recreated the directory in its final, globally available location, and ran Make again. Success! Apparently something in that compile process is looking to see what directory it is in and going back to it for data. I'd like to know what that is.

WSL permissions bits

Conflicts between Linux permissions and Windows permissions are a perennial problem for people who switch back and forth between WSL and Windows. One thing that helps is, when mounting a drive, to provide the -o metadata parameter to make sure that files have both Windows and Linux permission bits:

$ sudo mount -t drvfs g: /mnt/g -o metadata

Here's some good information about how WSL permissions work:

https://devblogs.microsoft.com/commandline/chmod-chown-wsl-improvements/

Research Computing and Data Capabilities Model

The capabilities model allow an institution to evaluate how well it supports various data and research requirements.

https://carcc.org/2020/07/09/announcing-the-rcd-cm-2020-community-data-participation-window/

What I found most interesting is what it refers to as the "five facings"

Researcher Facing Roles

Includes research computing and data staffing, outreach, and advanced support, as well as support in the management of the research lifecycle.

Examples: Research IT User Support, Research Facilitators, CI engineers, etc.

Data Facing Roles

Includes data creation; data discovery and collection; data analysis and visualization; research data curation, storage, backup, and transfer; and research data policy compliance.

Examples: Research Data Management specialists, Data Librarians, Data Scientists, etc.

 

Software Facing Roles

Includes software package management, research software development, research software optimization or troubleshooting, workflow engineering, containers and cloud computing, securing access to software, and software associated with physical specimens.

Examples: Research Software Engineers, Research Computing support, etc.

 

Systems Facing Roles

Includes infrastructure systems, systems operations, and systems security and compliance.

Examples: HPC systems engineers, Storage Engineers, Network specialists, etc.

 

Strategy and Policy Facing Roles

Includes institutional alignment, culture for research support,  funding, and partnerships and engagement with external communities. 

Examples: Research IT leadership 

Which, at the risk of sounding like a Myers-Briggs evaluation, seems to sum up nicely the important categories of staff in research computing.

Traversing a graph database with Gremlin

This is an invaluable tutorial on how to use the Gremlin query language to get results from a graph database. For some reason, all the internal links seem to be broken, but it's a ten-part series I think. Lesson six on projection and selection is particularly useful.

 https://www.datastax.com/blog/2017/09/gremlin-recipes-1-understanding-gremlin-traversals