From FarmShare

our GCE project ID is stanford.edu:barley-gce

I created a standard instance, it gets Ubuntu 12.04 by default. 3.7GB per core, up to 8 cores. ~7GB usable /tmp

https://developers.google.com/compute/docs/hello_world

Contents 1 minimum requirements 1.1 LDAP 1.2 GlusterFS 1.3 SGE 2 AFS 3 Kerberos 4 stanford packages

minimum requirements

• qmaster on senpai1 needs to be able to talk to execd on instance
• user information from ldap needs to be present on instance
• user data directory needs to be present on instance (e.g. /mnt/glusterfs)

Can compare to barley-tesq for LDAP settings.

LDAP

 sudo aptitude install ldap-utils libpam-ldap libnss-ldap

compare /etc/nsswitch.conf and /etc/ldap.conf to the one on barley-testq

test anonymous bind:

 ldapsearch -x -h ldap.stanford.edu -b "cn=accounts,dc=stanford,dc=edu" "(objectClass=*)"

The above command works from barley-testq but not from GCE instance, either firewall or IP ACL if I had to guess?

OK, give up on that for now.

More details, should work just fine if host has keytab: https://ikiwiki.stanford.edu/service/ldap/workgroup-pam-controls-without-puppet/

OK, it's a bit more complicated.

1. wallet create keytab host/176.91.59.108.bc.googleusercontent.com
2. wallet owner keytab host/176.91.59.108.bc.googleusercontent.com ADMIN
3. wallet -f /tmp/barley21-temp.keytab get keytab host/176.91.59.108.bc.googleusercontent.com

copy that file somehow to the instance:/etc/krb5.keytab

1. k5start -f /etc/krb5.keytab host/176.91.59.108.bc.googleusercontent.com@stanford.edu
2. ldapsearch -h ldap.stanford.edu -b cn=accounts,dc=stanford,dc=edu uid=whm

run

1. /usr/bin/k5start -b -p /var/run/nslcd/k5start_nslcd.pid -o nslcd -g nslcd -m 600 -f /etc/krb5.keytab -K 60 -u host/176.91.59.108.bc.googleusercontent.com -k /var/run/nslcd/ldap.tgt
2. service nslcd restart (k5start portion fails)
3. /usr/bin/k5start -b -p /var/run/nslcd/k5start_nslcd.pid -o nslcd -g nslcd -m 600 -f /etc/krb5.keytab -K 60 -u host/176.91.59.108.bc.googleusercontent.com -k /var/run/nslcd/ldap.tgt

GlusterFS

 sudo aptitude install glusterfs-client
 sudo mkdir -p /mnt/glusterfs

hmm, instance has only internal IP, and can't ping barley-storage01, let's look at that: https://developers.google.com/compute/docs/networking

The networking doc says any outgoing connection is allowed. Firewalls in the way:

• none on GCE side (outgoing)
• central firewall - project Stanford FarmShare, no incoming GlusterFS ports allowed, filed request
• iptables on barley-storage01, added to puppet iptables fragment
• gluster auth.allow, currently * (gluster volume info)

OK, waiting on firewall team, give up for now

Works fine after opening fw:

chekh@my-first-instance:~$ df -h
Filesystem                           Size  Used Avail Use% Mounted on
/dev/vda1                            9.4G  1.1G  7.9G  12% /
none                                 1.9G  4.0K  1.9G   1% /dev
none                                 378M  128K  377M   1% /run
none                                 5.0M     0  5.0M   0% /run/lock
none                                 1.9G     0  1.9G   0% /run/shm
barley-storage01.stanford.edu:/bvol  3.6T  2.7T  757G  79% /mnt/glusterfs

SGE

 sudo aptitude install gridengine-client gridengine-exec
 echo "senpai1.stanford.edu" > /var/lib/gridengine/default/common/act_qmaster

Instance needs to be able to talk to qmaster over TCP 6444:

 barley-testq:/root# qping -info senpai1 6444 qmaster 1

qmaster needs to be able to talk to instance over tcp 6445:

 gcutil addfirewall allowge --description="Allow qmaster on senpai1 to interrogate sge_execd." --allowed="tcp:6445" 

Actually that allows TCP 6445 from everywhere, but I guess that's fine for now, needed --allowed-ip-sources=IP-OF-senpai1

on senpai1:

 qping -info 108.59.91.176 6445 execd 1

AFS

TODO: install openafs packages and copy /etc/openafs config to set AFS cell

Kerberos

TODO: install correct packages and copy over krb5.conf and generate host keytab and auks service keytab

stanford packages

TODO: install packages stanford-server-timeshare, auks, stanford-ldap-tools, openafs-whatever pre-requisite: configure correct repo Tried /etc/apt/sources.list.d/stanford.list, but get 403 Forbidden from outside of Stanford. Giving up for now.

Russ added 108.59.80.0/20 to the local repo IP ACL, and also need to 'sudo aptitude install stanford-keyring' to get the GPG keys.