Pxe toucheup by CarlFK · Pull Request #3 · CarlFK/video-stack-deploy

CarlFK · 2017-02-20T23:16:23Z

base that can be used as is to build production boxes, and extended for R&D of new and and untested things.

stefanor · 2017-02-20T23:25:56Z

roles/apt/templates/30apt-proxy.j2

@@ -1,2 +1,2 @@
 // {{ ansible_managed }}
-Acquire::http::Proxy "{{ apt_proxy }}";
+Acquire::http::Proxy "http://{{ inventory_hostname }}:8000/";


I can't see how that's correct. Not every machine runs an apt proxy.

The file doesn't exit if there is no proxy

38 - name: setup apt proxy
39 template:
40 src: templates/30apt-proxy.j2
41 dest: /etc/apt/apt.conf.d/30apt-proxy
42 when: apt_proxy != ""

42 when: apt_proxy != ""

but that isn't right.

What I'm getting at is, inventory_hostname is the local machine's hostname. So every machine will be pointing at itself as an APT proxy. That can't be right.

Ohh... right. Hmm....
I am used to all these files living on the pxe server as static files.
I was thinking ansible creates the static file on the pxe server and then the other machines pull a copy.
but that isn't how things work here.

this bit is
31 # d-i puts the apt config in apt.conf, but we prefer a .d file

d-i puts the correct value in apt.conf, how about we just leave it there?

or patch d-i :p

stefanor · 2017-02-20T23:26:42Z

roles/dhcp-server/templates/local.conf.j2

 dhcp-authoritative
 enable-tftp
 tftp-root=/srv/tftp
+no-hosts


If we're using our static hosts stuff, then we'll probably want to serve those over DNS too. wouldn't we?

that's a bit of a contradiction. or redundancy. it is either addressing a case that will never exist, or addressing the case where things are broken. If /etc/hosts is in place, then we don't need dns.

however, even with static, dns can serve everything by putting a 2nd almost copy of the data in a file. almost = different IP for $server

I like putting the name:IP data in it's own file that is just for the dns, not also /etc/hosts.

Mainly because of waning $server:IP to be defined as something other than 127.0.1.1 like it is in /etc/hosts.

or redundancy

That. You saw how it was useful at LCA, when we didn't have a global network, yet. Things still worked inside rooms. And we could ssh into things by hostname, because there was DNS with this stuff in it.

however, even with static, dns can serve everything by putting a 2nd almost copy of the data in a file. almost = different IP for $server

Sure. But we can just solve all of this by changing the server's IP in its own /etc/hosts.

changing the server's IP in its own /etc/hosts

I went down that path and got angry :p

d-i builds this:
juser@dc10b:~$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 dc10b

is ...1.1 needed?

I made a task using lineinfile but my code kept adding additional "ip \t host" lines each time I re-ran. so I switched to not using etc/hosts and using a separate file for dnsmasq and was happy.

I have mixed feelings about dnsmasq using etc/hosts.

give me the ansible to just solve all of this and I'll give it a whirl

dns can serve everything ...
Let's do that.

This only effects the dhcp server. it separates the dns config from the servers etc/hosts, andI am ok with that.

replacing "127.0.1.1 dc10b" is too hard for me, and I don't see much benefit.

I still don't like this change, I find it useful to be able to ssh into machines by hostname, on the video network.

stefanor · 2017-02-20T23:27:02Z

roles/nfs-server/templates/exports.j2

 # fsid=X is needed for XFS filesystems with 64bit inodes
-/srv/{{ nfs_server }}/veyepar {{ eth_local_ip_address|default(ansible_default_ipv4.network + '/' + ansible_default_ipv4.netmask) }}(ro,async,no_subtree_check,fsid=0)
-/srv/{{ nfs_server }}/video {{ eth_local_ip_address|default(ansible_default_ipv4.network + '/' + ansible_default_ipv4.netmask) }}(rw,sync,no_subtree_check,fsid=1)
+{% if eth_local_ip_address is defined %}


Surely it'll always be defined?

I was trying to duplicate what the old code was doing:
{{ eth_local_ip_address|default(...

which made me wonder what the point was.

Ah, yeah. I probably didn't realise there was a default defined.

stefanor · 2017-02-20T23:27:32Z

roles/tftp-server/defaults/main.yml

 netboot_image: https://deb.debian.org/debian/dists/stretch/main/installer-amd64/current/images/netboot/netboot.tar.gz
-late_command_url: https://anonscm.debian.org/cgit/debconf-video/ansible.git/plain/setup_ansible.sh
+# late_command_url: https://anonscm.debian.org/cgit/debconf-video/ansible.git/plain/setup_ansible.sh
+late_command_url: http://{{ inventory_hostname }}/lc/late.sh


We could use TFTP, and avoid having to setup a web server

I find tftp harder to do R&D

wget is so much easier to use and read later than the curl command needed for tftp.

stefanor · 2017-02-20T23:27:59Z

roles/tftp-server/files/late_command.sh

+# We clone our ansible repository and copy the ansible config files
+
+git clone https://anonscm.debian.org/git/debconf-video/ansible.git /root/debconf-ansible
+git clone https://github.com/xfxf/lca2017-av.git /root/lca2017-av


That, obviously, can't be committed.

ah right..
I knew something wasn't quite right, but thought I was doing the equivalent thing just changing the host from github to local box.
but the whole late_command.sh name/url was a config option, and now that is gone.
Hmm...

stefanor · 2017-02-20T23:30:59Z

roles/dhcp-server/templates/local.conf.j2

 tftp-root=/srv/tftp
+no-hosts
+host-record={{ inventory_hostname }},{{ eth_local_ip_address }}
+


stefanor · 2017-02-20T23:31:30Z

roles/tftp-server/tasks/main.yml

-    line: \1 auto=true interface=auto url=tftp://10.20.0.1/preseed.cfg \2
+    line: \1 auto=true interface=auto url=http://{{ inventory_hostname }} \2
  with_items: "{{ menus.files }}"
+


stefanor · 2017-02-20T23:32:00Z

roles/tftp-server/templates/preseed.cfg.j2

 d-i mirror/http/directory string /ubuntu
 {% endif %}
-d-i mirror/http/proxy string {{ apt_proxy }}
+d-i mirror/http/proxy string{% if apt_proxy %}http://{{ inventory_hostname }}:8000/{% endif %}


apt_proxy is now boolean?

correct.
I don't think you can reference inventory_hostname in the inventory/group_vars/all.yml

stefanor · 2017-02-20T23:32:19Z

roles/tftp-server/templates/preseed.cfg.j2


 # Individual additional packages to install
-d-i pkgsel/include string openssh-server curl
+d-i pkgsel/include string openssh-server curl python python3


for ansible. not sure if it wanted v2 or 3.

moved from late_command.sh apt install python

I think it is a safe assumption that we:
want the box ready for ansible. (all cases)
may not use late_command.sh (some cases)

Ansible has dependencies declared, you don't need to do this.

I think you are saying I should do:

d-i pkgsel/include string openssh-server curl ansible

but ansible doesn't need to be installed on it's target

personally, I don't mind installing everything everywhere ;)

stefanor · 2017-02-20T23:32:49Z

roles/tftp-server/templates/preseed.cfg.j2

 d-i preseed/late_command string in-target sh -c "curl -o penultimate_setup.sh '{{ late_command_url }}' && ANSIBLE_UNDER_DI=1 sh penultimate_setup.sh && rm penultimate_setup.sh"
 {% else %}
-d-i preseed/late_command string in-target sh -c "wget -O penultimate_setup.sh '{{ late_command_url }}' && ANSIBLE_UNDER_DI=1 sh penultimate_setup.sh && rm penultimate_setup.sh"
+# d-i preseed/late_command string in-target sh -c "wget -O penultimate_setup.sh '{{ late_command_url }}' && ANSIBLE_UNDER_DI=1 sh penultimate_setup.sh && rm penultimate_setup.sh"


Similar comment about commenting stuff out, as before.

stefanor · 2017-06-14T04:29:13Z

roles/apt/tasks/main.yml

    dest: /etc/apt/sources.list
  notify: apt update
-  when: ansible_distribution == 'Debian'
+  when: distro == 'Debian'


I don't understand the point of these changes

stefanor · 2017-06-14T04:33:19Z

roles/dhcp-server/templates/local.conf.j2

 tftp-root=/srv/tftp
+no-hosts
+host-record={{ inventory_hostname }},{{ eth_local_ip_address }}
+host-record={{ inventory_hostname }}.{{ domain }},{{ eth_local_ip_address }}


What's the point of having the machine return itself in DNS, but nothing else?

stefanor · 2017-06-14T04:33:57Z

roles/dhcp-server/templates/local.conf.j2

 dhcp-authoritative
 enable-tftp
 tftp-root=/srv/tftp
+no-hosts


I still don't like this change, I find it useful to be able to ssh into machines by hostname, on the video network.

stefanor · 2017-06-14T04:34:28Z

roles/nfs-server/templates/exports.j2

 # fsid=X is needed for XFS filesystems with 64bit inodes
-/srv/{{ nfs_server }}/veyepar {{ eth_local_ip_address|default(ansible_default_ipv4.network + '/' + ansible_default_ipv4.netmask) }}(ro,async,no_subtree_check,fsid=0)
-/srv/{{ nfs_server }}/video {{ eth_local_ip_address|default(ansible_default_ipv4.network + '/' + ansible_default_ipv4.netmask) }}(rw,sync,no_subtree_check,fsid=1)
+{% if eth_local_ip_address is defined %}


stefanor · 2017-06-14T04:34:56Z

roles/opsis/files/minirc.dfl

@@ -0,0 +1,8 @@
+pu port             /dev/ttyACM0


We have better port names than that, provided by our udev package

stefanor · 2017-06-14T04:37:07Z

roles/tftp-server/defaults/main.yml

 ---
 netboot_image: https://deb.debian.org/debian/dists/stretch/main/installer-amd64/current/images/netboot/netboot.tar.gz
-late_command_url: https://anonscm.debian.org/cgit/debconf-video/ansible.git/plain/setup_ansible.sh
+# late_command_url: https://anonscm.debian.org/cgit/debconf-video/ansible.git/plain/setup_ansible.sh


Please delete rather than comment out. git has history.

stefanor · 2017-06-14T04:38:02Z

roles/tftp-server/files/late_command.sh

+# We clone our ansible repository and copy the ansible config files
+
+# git clone https://anonscm.debian.org/git/debconf-video/ansible.git /root/debconf-ansible
+git clone https://github.com/CarlFK/video-stack-deploy.git /root/debconf-ansible


This, and many other things in this file, should be templated.

stefanor · 2017-06-14T04:40:25Z

roles/tftp-server/tasks/main.yml

+  apt:
+    name: "{{ item }}"
+  with_items:
+  - nginx


It's a list of one item, just install it, no need to iterate over the list.

stefanor · 2017-06-14T04:43:57Z

roles/xorg/tasks/lightdm.yml

+  file:
+    path: /etc/lightdm/lightdm.conf.d
+    state: directory
+  when: distro=="Debian"


Do we need that? Surely if it already exists, it's no big deal?

stefanor · 2017-06-14T04:44:48Z

roles/apt/tasks/main.yml

    path: /etc/apt/apt.conf.d/30apt-proxy
    state: absent
-  when: apt_proxy == ""
+  when: not apt_proxy


I assume you can delete this whole block, because nothing is ever going to create that file any more.

stefanor · 2017-07-09T02:27:20Z

roles/tftp-server/defaults/main.yml

-pw_salt: U1ESiTXUQE0tFh2r4LRl
+# how to make the hash:
+# $ printf "r00tme" | mkpasswd -s -m md5
+user_password_hash: $1$G1uRZoLB$r68rPXmpJl/C6ervlynKa/


The default doesn't belong here, it should probably default to null here (and have an appropriate if block in the preseed template)

stefanor · 2017-07-09T02:27:38Z

roles/tftp-server/defaults/main.yml

-user_password: null
-pw_salt: U1ESiTXUQE0tFh2r4LRl
+# how to make the hash:
+# $ printf "r00tme" | mkpasswd -s -m md5


SHA-512 please

stefanor · 2017-07-09T02:28:51Z

roles/tftp-server/defaults/main.yml

+
 user_name: videoteam
-user_password: null
-pw_salt: U1ESiTXUQE0tFh2r4LRl


This change needs to be made in more places (inventory/group_vars/all, roles/users/defaults/main.yml)

…l debian nodes

stefanor reviewed Feb 20, 2017

View reviewed changes

stefanor reviewed Jun 14, 2017

View reviewed changes

stefanor mentioned this pull request Jul 9, 2017

Make apt_proxy a boolean #11

Merged

stefanor force-pushed the pxe-toucheup branch 3 times, most recently from a5386c7 to 919da4b Compare July 9, 2017 00:48

stefanor mentioned this pull request Jul 9, 2017

NFS server should listen on local ip #12

Merged

stefanor force-pushed the pxe-toucheup branch 2 times, most recently from 6accdcd to b9c9955 Compare July 9, 2017 02:07

stefanor reviewed Jul 9, 2017

View reviewed changes

stefanor mentioned this pull request Jul 9, 2017

Use crypted password rather than plain text #13

Merged

stefanor force-pushed the pxe-toucheup branch from b9c9955 to 2e26555 Compare July 9, 2017 22:34

This was referenced Jul 9, 2017

Restart after changing network configuration files #14

Merged

Addd support for minicom #15

Merged

stefanor force-pushed the pxe-toucheup branch from 2e26555 to 7bdda8a Compare July 10, 2017 00:02

stefanor mentioned this pull request Jul 10, 2017

Serve preseed and late_command over http #17

Merged

CarlFK added 12 commits July 9, 2017 19:52

smooth over rough edges to make testing and extension easier

f6248d9

fix missing space typo

0c6df08

tweak the regex to only catch the first append line in txt.cfg.

064e19b

setup preseed for both jessie and xenial (sort of).

7a7757f

use CarlFK's pending PR repos

9f3f49a

use default protocol (httl) and max the installer debug level

cc25503

Debian installer constructs FQDN, so resolve it too.

5eae9b1

hammer in ansible 2.2, make elegant later.

783185c

Fix line-oh.

0e3e70a

use correct git command to checkout a branch.

8c64cc7

only test dc roles

d4a102b

move disk=hda to boot params for on the fly adjustment.

f9671dc

CarlFK added 9 commits July 9, 2017 19:52

sda is a better default than hda.

6ef063b

grub-ipxe makes testing pxe easier

3275ffa

chagne ansible_distribution to distro so that a Ubuntu box can instal…

fc635af

…l debian nodes

stretch not jessie

157e33c

ansible in main repos is 2.2.x, that's good.

42025c6

Add dupport for minicom

f018e67

run preseed so we cna see stdout.

a5d09fe

moved things to be easeir to read

c54c339

default and document distro var

fd59a81

stefanor force-pushed the pxe-toucheup branch from 7bdda8a to fd59a81 Compare July 10, 2017 02:53

stefanor force-pushed the master branch from 265129e to fd25ddd Compare November 11, 2017 03:19

Conversation

CarlFK commented Feb 20, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment