Enzyklopädie

Gephi Workshop EPFL @DH2020

• Watch the Keynote: **Conspiracy in the Time of Covid: A Computational Pipeline for the Discovery of Conspiracy Narrative Frameworks. Tim Tangherlini (University of California – Berkeley). (There is also an interview with Tim available here.)
• Watch the example study: Visualising the Merce Cunningham Company Network. Clarisse Bardiot (UPHF).
• Watch the workshop prepatory videos:
  • How to Install Gephi 0.9.2 (also on GitHub),
  • First Contact with Gephi 0.9.2, and
  • Watch Layouts, Clusters, and Where to Find Them.
  • Download and install Gephi.

Ressources: https://github.com/CultureAnalytics/DH2020

Create and Manage Security Groups

You can create new security groups to provide certain servers with specific access. After login with the user credential, follow the steps:

1. 1. From the left menu bar, select Project→Network→Security Groups and then click “Create Security Group”.
   2. Determine a unique meaningful name (Ex. Goenet) and an optional description then click “Create Security Group”.

Once the security group has been created successfully, it will be shown in the list with an option to manage its security rules.

Manage Security Rules

You can add/remove security rules to/from a security group, in order to change the provided access. Click “Manage Rules” from the very right of the security group row. A form containing all defined rules will be shown. By default, all ports are opened for outbound connections and no inbound connections are allowed. As a result, you can see that it contains 2 Egress rules whit no limitation on IPs and ports.

1. 1. To add a new rule, click “Add Rule” on top-right of the form.
   2. To define Goenet rule, first select TCP protocol from “Rule” drop-down list.
   3. Leave the “Direction” field with the value Ingress (inbound connections).
   4. For TCP protocol select All ports from “Open Ports” drop-down list.
   5. Leave the “Remote” field with the value CIDR, and in “CIDR” field specify the GWDG/University’s IP range: 134.76.0.0/16. The default value “0.0.0.0/0” means from any IP (no protection) .
   6. Then click “Add” on bottom-right of the form.

Assign Security Group to Server

From the left menu bar, select Project → Compute → Instances and then click the small arrow on the very right of each instance row called “Actions”. After that select “Edit Security Groups” then click the “+” (add sign) of “Goenet” from Security Groups list. Then click “Save” on bottom-right of the form to apply the rules on the server.

Source:  Create and Manage Security Groups

GRO.data ist ein universelles Forschungsdatenrepositorium für den Göttingen Campus. Forschende mit Zugehörigkeit zu einer Institution des Göttingen Campus können es kostenfrei nutzen. GRO.data dient verschiedenen Zwecken, wie z.B.:

• Datensätze auf einfache Art dauerhaft sichern
• Änderungen an Datensätzen über mehrere Versionen hinweg verfolgen
• Daten mit KollegInnen teilen
• Daten öffentlich verfügbar machen (publizieren)
• Persistente Identifikatoren für publizierte Daten erhalten

Wer kann den Dienst nutzen?

Jede/r Forschende am Göttingen Campus kann ihr/sein institutionelles Konto nutzen um sich einzuloggen, einen persönlichen Bereich („Dataverse“) anzulegen und dann hierin Dateien als „Datasets“ hochzuladen. Sie können dann anderen Forschenden oder Gruppen Zugriff auf diese Dateien ermöglichen oder die Daten publizieren und einen DOI erhalten, um diesen bspw. für die Zitation zu nutzen. Wenn Sie eine ORCID iD haben, können Ihre Daten auch automatisch zu Ihrem ORCID record hinzugefügt werden.

Technische Details

GRO.data basiert auf dem Open-Source-Programm Dataverse. Es wurde und wird kontinuierlich von der Göttinger eResearch Alliance angepasst, um sich gut in die lokale Infrastruktur zu integrieren und den Zugriff über Single Sign-On, d.h. mit Ihrem GWDG-Konto, zu ermöglichen. Die GWDG hostet diesen Dienst und kümmert sich um Datensicherheit und -schutz, einschließlich Backups und Zugriffskontrolle, sowie um die Verfügbarkeit und Zuverlässigkeit.

grab-site is the archivist’s web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

GitHub repo: https://github.com/ArchiveTeam/grab-site

Installation steps:

Install on Ubuntu 18.04, 20.04, 22.04, Debian 10 (buster), Debian 11 (bullseye)

1. On Debian, use su to become root if sudo is not configured to give you access.

sudo apt-get updatesudo apt-get install –no-install-recommends \    wget ca-certificates git build-essential libssl-dev zlib1g-dev \    libbz2-dev libreadline-dev libsqlite3-dev libffi-dev libxml2-dev \    libxslt1-dev libre2-dev pkg-config

If you see Unable to locate package, run the two commands again.

2. As a non-root user:

wget https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installerchmod +x pyenv-installer./pyenv-installer~/.pyenv/bin/pyenv install 3.8.15~/.pyenv/versions/3.8.15/bin/python -m venv ~/gs-venv~/gs-venv/bin/pip install –no-binary lxml –upgrade git+https://github.com/ArchiveTeam/grab-site

–no-binary lxml is necessary for the html5-parser build.

3. Add this to your ~/.bashrc or ~/.zshrc:

PATH=“$PATH:$HOME/gs-venv/bin“

and then restart your shell (e.g. by opening a new terminal tab/window).

Upgrade an existing install

To update grab-site, simply run the ~/gs-venv/bin/pip install … or nix-env … command used to install it originally (see above).

After upgrading, stop gs-server with kill or ctrl-c, then start it again. Existing grab-site crawls will automatically reconnect to the new server.

Using grab-site

First, start the dashboard with:

gs-server

and point your browser to http://127.0.0.1:29000/

Note: gs-server listens on all interfaces by default, so you can reach the dashboard by a non-localhost IP as well, e.g. a LAN or WAN IP. (Sub-note: no code execution capabilities are exposed on any interface.)

Then, start as many crawls as you want with:

grab-site ‚URL‘

>>Do this inside tmux unless they’re very short crawls.<<

grab-site outputs WARCs, logs, and control files to a new subdirectory in the directory from which you launched grab-site, referred to here as „DIR“. (Use ls -lrt to find it.)

You can pass multiple URL arguments to include them in the same crawl, whether they are on the same domain or different domains entirely.

See „SolrWayback“ entry in the Knowledge Base for help with accessing WARC files.

• https://info.gwdg.de/dokuwiki/doku.php?id=de:services:server_services:gwdg_cloud_server:manual_v3