Skip to content

README.md

Latest commit

 

History

History
347 lines (252 loc) · 12.1 KB

README.md

File metadata and controls

347 lines (252 loc) · 12.1 KB

External Developers

You can use the deken command line tool to create packaged zipfiles with the correct searchable architectures in the filename, for example freeverb~-v0.1-(Linux-amd64-64)-externals.zip.

If you don't want to use the deken packaging tool you can zip and upload the files yourself. See the "Filename format" section below.

Get started

Prebuilt Binaries

If you don't want to install Python3, bash (MSYS),... as described below, you can also download self-contained binaries from our Continuous Integration setup:

If they don't work for you, you might want to check the releases page for downloads that have been tested by humans.

These builds are snapshots of the latest development branch of deken.

On Debian and derivatives (like Ubuntu), deken is also readily available via apt:

apt-get install deken

Docker containers

docker is all the rage these days, so naturally there is a docker image for deken as well.

Get the latest and greatest release:

docker pull registry.git.iem.at/pd/deken

(For the more daring, you can also grab the lastest development snapshot under the main tag).

To use it to create your packages:

$ ls -d deken-test*
deken-test/

$ docker run --rm -ti                      \
    --user $(id -u) --volume $(pwd):/deken \
    registry.git.iem.at/pd/deken           \
    deken package --version 1.2.3 deken-test

$ ls -d deken-test*
deken-test/
'deken-test[v1.2.3].dek'
'deken-test[v1.2.3].dek.sha256'
$

And to upload packages:

docker run --rm -ti                        \
    --user $(id -u) --volume $(pwd):/deken \
    --env DEKEN_USERNAME=mydekuser         \
    registry.git.iem.at/pd/deken           \
    deken upload *.dek

GPG-signing with Docker

Within the container, deken will not attempt to GPG-sign your packages by default. If your container has access to your GPG keys, you can enable signing by passing the --sign-gpg flag to package (resp. upload).

The following assumes that you have a properly configured GPG setup in your ~/.gnupg, and gpg-agent is running on your host machine:

docker run --rm -ti                        \
    --user $(id -u) --volume $(pwd):/deken \
    --volume ${HOME}/.gnupg/:/.gnupg/:ro --volume /run/user/$(id -u)/:/run/user/$(id -u)/:ro \
    registry.git.iem.at/pd/deken           \
    deken package --sign-gpg --version 1.2.3 deken-test

Manual bootstrap

$ mkdir -p ~/bin/
$ curl https://raw.githubusercontent.com/pure-data/deken/main/developer/deken > ~/bin/deken
$ chmod 755 ~/bin/deken
$ deken
This is your first time running deken on this machine.
I'm going to install myself and my dependencies into ~/.deken now.
Feel free to ctrl-C now if you don't want to do this.
...

See config.md for deken's configuration file format.

If you get an error like

-bash: deken: command not found

then make sure that ~/bin is in your PATH.

Prerequisites

deken requires Python3 to be installed on your computer (and available from the cmdline). You can test whether python3 is installed, by opening a terminal and running python3 --version.

For installing (and updating) deken, you will also need curl (or wget) for downloading from the cmdline.

macOS

On macOS, you can install missing dependencies with brew. Once you have installed brew, run the following in your terminal:

brew install python3
Windows

On Windows you might need to install MSYS2/MinGW64, which comes with pacman as a package manager to install missing dependencies. Once you have installed pacman, run the following in your terminal:

pacman -Suy python3

Show help

$ deken -h

Upgrade

To run a self-upgrade (not supported on all platforms), simply do:

$ deken upgrade

Create and Upload a package

You have a directory containing your compiled externals object files called my_external.

This command will create a file like my_external[v0.1](Linux-amd64-64).dek and upload it to your account on https://puredata.info/ where the search plugin can find it:

$ deken package -v 0.1 my_external
$ deken upload "my_external[v0.1](Linux-amd64-64).dek"

You can also just call the 'upload' directly and it will call the package command for you in one step:

$ deken upload -v 0.1 my_external

The upload step will also generate a .sha256 checksum file and upload it along with the dek file. If possible, also a GPG signature file (with the .asc extension) will be created and uploaded (but you must have GPG installed and you need to have a GPG key for signing. The GPG signature mostly makes sense, if your GPG key is cross-signed by (many) other people).

Creating/Uploading packages on a different machine

deken inspects the files in the directory to determine the target platform (rather than just checking on which system you are currently running). Therefore, if it is not feasible to install deken on the machine used for building your Pd library, you can run deken on another machine,

Example: You build the "my_external" library on OSX-10.5, but (due to OSX-10.5 not being supported by Apple anymore) you haven't installed deken there. So you simply transfer the "my_external" directory to your Linux machine, where you run deken package my_external and it will magically create the my_external[v3.14](Darwin-i386-32)(Darwin-amd64-32)-externals.tgz file for you, ready to be uploaded.

Filename format

The deken tool names a zipfile of externals binaries with a specific format to be optimally searchable on puredata.info;

LIBNAME[vVERSION]{(ARCH)}.dek
  • LIBNAME is the name of the externals package ("zexy", "cyclone", "freeverb~").
  • VERSION contains the version information for the end use (this information is optional though strongly encouraged)
  • ARCH is the architecture specifier, and can be given multiple times (once for each type of architecture the externals are compiled for within this archive). It is either "Sources" (see below or OS-MARCH-BIT, with:
    • OS being the Operating System. Typical values are:
      • Linux
      • Darwin
      • Windows
    • MARCH is the machine architecture, e.g.:
      • i386 (32bit Intel/AMD-compatible CPUs)
      • amd64 (64bit Intel/AMD-compatible CPUs; synonymous for x86_64, though amd64 is the preferred form)
      • ppc (the PowerPC architecture popular in old Apple computers)
      • armv7l (little-endian 32bit ARM CPUs as found in the Raspberry Pi 3)
    • BIT is the size of Pd's numbers in bits (usually 32; for double-precision it will be 64)

Note that the archive should contain a single directory at the top level with NAME the same as the externals package itself. For example a freeverb~ externals package would contain a directory "freeverb~" at the top level of the zipfile in which the externals live.

The version string must be enclosed by square brackets ([]) and start with a v. The version string itself must not contain any brackets or parentheses. Strictly speaking, the version (with the enclosing brackets) is optional, however it is highly suggested that you provide it.

The curly braces around the "(ARCH)" specifiers are only to indicate that this section can occur multiple times (or not at all). However, the round parentheses "()" enclosing the architectures string must be included to separate the architectures visibly from each other.

In plain English this means:

the library-name, followed by an optional version string (starting with [v and ending with ]), followed by zero or more architecture specifications (each surrounded by (parentheses)), and terminated by .dek.

Here is the actual regular expression used:

(.*/)?([^\[\]\(\)]+)(\[v[^\[\]\(\)]+\])?((\([^\[\]\(\)]+\))*)\.(dek)

with the following matching groups:

  • #0 anything before the path (always empty and ignored)
  • #1 = path to filename (ignored)
  • #2 = library name
  • #3 = options (including the version)
  • #4 = archs
  • #5 = last arch in archs (ignored)
  • #6 = extension ('dek')

Some examples:

adaptive[v0.0.extended](Linux-i386-32)(Linux-amd64-32).dek
adaptive[v0.0.extended](Sources).dek
freeverb~(Darwin-i386-32)(Darwin-x86_64-32)(Sources).dek
list-abs[v0.1].dek

Sourceful uploads

deken is very much about sharing. To make sharing a more lasting experience, deken encourages the upload of "source-packages" besides (pre-compiled) binary packages.

This is especially important if you are uploading a library that has been released under a license that requires you to share sources along with binaries (e.g. software licensed under the Gnu GPL), where it is your obligation to provide the source code to the end users. In other situations, having Source packages might be less important (e.g. it is fine to use deken with closed source libraries), however we would like to encourage sharing of sources.

The way deken implements all this is by using a special pseudo architecture "Sources", which contains the sources of a library.

deken package tries to automatically detect whether a package contains Sources by looking for common source code files (*.c, *.cpp, ...).

When uploading a package, deken will ensure that you are also uploading a Source package of any library. If a Source package is missing, deken will abort operation. You can override this (e.g. because you have already uploaded a Source package; or because you simply do not want to upload any sources) by using the --no-source-error flag.

For uploading a Source package along with binary packages, you can upload one package file with multiple archs (including a "Sources" arch) or multiple package files (one for the "Sources" arch).

$ deken upload frobnozzel(Windows-i386-32)(Sources).dek
$ deken upload foobar[v0.1](Linux-x86_64-32).dek foobar[v0.1](Sources).dek

objectlists

Sometimes the user only knows the object they need, not the library. Therefore, a search initiated via the deken-plugin (Pd's package manager) also searches for objects. For this to work, the infrastructure must know which objects are contained in a library; this is done via an objectlist file.

The objectlist file consists of exactly one line per object, with the object-name at the beginning, followed by a TAB (\t) and a short (single-line) description of the object.

frobnofy	frobfurcate a bugle of numbers
frobnofy~	signal frobfurcation

The objectlist file has the same name as the package with a .txt appended. E.g. if your library is called frobnozzel(Windows-i386-32)(Sources).dek, the objectlist file would have the name frobnozzel(Windows-i386-32)(Sources).dek.txt This file must be uploaded to the same directory as the .dek file.

deken will try to automatically generate an objectlist file for a package. It looks for all "*-help.pd" files in the library directory, and creates an entry in the objectlists for each. The short description is taken from the DESCRIPTION comment in the [pd META] subpatch within the help-patch. If no DESCRIPTION comment can be found, a generic description is used.

You can provide your own (manually maintained) objectlist file via the --objects flag:

$ deken package --objects mylist.txt my_external

To prevent the creation/use of an objectlist file, pass an empty string

$ deken package --objects "" my_external

In general, it is preferable if the description of the object in the META subpatch is included in the object's help file (and let deken generate the objectlist from it), as this allows others (humans, Pd, plugins, ...) to access the same information as well.

Troubleshooting

see DEVELOPMENT.md for some troubleshooting advice.