• VR re:stream

  [[https://gitlab.com/voicerepublic/voicerepublic_logorrhoe/badges/master/build.svg]]

** General

This is literate documentation for the VR native streaming desktop application. The stack is based on

• [[http://clojure.org/][Clojure]]
• [[https://github.com/daveray/seesaw][seesaw]] [a LISPy Swing wrapper]

*** Audio Data Flow

#+BEGIN_SRC artist +----------------------+ +--------------------+ +-------------------+ | Audio Input | | MP3 Encoding | | Web Audio Stream | | Java sampled +------->| shell lame +--------->| HTTP PUT | +----------------------+ +--------------------+ +-------------------+ #+END_SRC

To be able to stream Audio date from /sampled/ over to the Shell (/lame/) and from there again to /http put/, many options were evaluated. The Clojure implementation of /clojure.java.shell/ does not support creating or sustaining a stream of output, only discrete steps. See [[http://clojuredocs.org/clojure.java.shell/sh][ClojureDoc]] for more information.

**** Alternative options

***** Using /clojure.java.shell2/

This is the version that is currently implemented!

According to [[https://groups.google.com/forum/#!topic/clojure-dev/A6xFhcPKdws][this discussion]] on the Clojure dev mailing list, Stuart Halloway (who is very active on this mailing list) sees /clojure.java.shell/ as under-powered and doesn't want to add any more effort in sustaining it's development.

However, Marc Limotte has created /clojure.java.shell2/ which is an extension of the core library and supports various non-breaking additions.

***** Encoding in discrete steps

This has several obvious downsides. For example that it would create multiple MP3 blobs, each with it's own header. Also synchronization between sending those blobs would be hard.

***** Monkey Patching /clojure.java.shell/

I tried, but got stuck with deadlocks. I didn't continue down that path, because other people have spent more time on doing this right (see below).

***** Using the [[https://github.com/Raynes/conch][Conch Library]] for shelling out asynchronously

This one was a good contender, however it was dropped in favor of /clojure.java.shell2/. The reasons being that /shell2/ feels more idiomatic and that Conch isn't being actively developed anymore. /shell2/ hasn't seen a commit even longer, but is based on the Clojure core lib /clojure.java.shell/ which gives it some additional credibility.

***** Using Java ProcessBuilder Java has a Class for creating and managing Sub-Processes called [[http://docs.oracle.com/javase/8/docs/api/java/lang/ProcessBuilder.html][ProcessBuilder]]. I did some initial tests, but /shell2/ prevailed.

** Icecast

*** Stats

• http://0.0.0.0/admin/stats.xml
• http://0.0.0.0/status-json.xsl

*** Protocol Reverse engineered streaming of Ogg Vorbis files:

#+BEGIN_SRC clojure (client/put "http://52.58.65.224/4609" { :basic-auth ["source" "thisisagoodpassword"] :body (clojure.java.io/file "/home/munen/src/voicerepublic_icecast_tests/manual_put/test.ogg") :headers { :user-agent "vr_shout/0.2.0" :ice-bitrate "128" :content-type "application/ogg" :ice-name "VR Server Name" :ice-genre "Rock" :ice-title "VR Title" :ice-url "https://voicerepublic.com" :ice-private "0" :ice-public "1" :ice-description "VR Server Description" :ice-audio-info "ice-samplerate=44100;ice-bitrate=128;ice-channels=2" } })

#+END_SRC

** To-dos ** Implementation Notes *** Finding available audio input devices Note: It would be possible to check for LINE_IN and MICROPHONE directly using (AudioSystem/isLineSupported Port$Info/LINE_IN) and (AudioSystem/getLine Port$Info/LINE_IN) However, this does not always return a line with said capability (seen in Debian 8). Therefore, we're using the method to request a line via the Mixer.

*** Why streaming MP3 and not Vorbis?

[[http://svn.xiph.org/trunk/vorbis-java/][vorbis-java]] is the official Java lib from Xiph, the creators of Ogg Vorbis. It has example source and even a port of the libshout lib.

Unfortunately, though, it hasn't been updated since 2007, is still in Beta and not available from Maven. For a manual install, package the Class files with jar cf something.jar [files] and then create a local Maven Repo for Leiningen: http://www.elangocheran.com/blog/2013/03/installing-jar-files-locally-for-leiningen-2/

*** libshout-java

• [[https://github.com/OlegKunitsyn/libshout-java][libshout-java]]

**** Install

Note that putting a Jar into /lib or using :resource-paths in project.clj doesn't seem to be the way to go since Leiningen 2. ***** OS X ****** Checkout libshout-java #+BEGIN_SRC shell git clone git@github.com:poochiethecat/libshout-java.git cd libshout-java brew install ./libshout.rb mvn install cp ./target/libshout-java.so ~/src/voicerepublic_logorrhoe/target/libshout-java.so #+END_SRC

***** Debian ****** Checkout libshout-java

#+BEGIN_SRC shell apt-get install git libshout3-dev maven git clone https://github.com/OlegKunitsyn/libshout-java.git cd libshout-java #+END_SRC

****** Fix a test (otherwise it will not install)

• Fix the test /testVersion()/ in /src/test/java/com/gmail/kunicins/olegs/libshout/LibshoutTest.java/ to check for your installed version of libshout
• It's tested and working with version 2.3.1
• Find your version via

#+BEGIN_SRC shell apt-cache show libshout3-dev | ag version | egrep -o "2.[0-9].[0-9]" #+END_SRC

• Then install the library to your local maven repository so that lein deps can find it

#+BEGIN_SRC shell mvn install #+END_SRC

***** TODO Move this compiled Library to a HTTP repo so that not every user has to install it to a local repo

** Development

*** Package for OS X

To start of packaging the Java Swing GUI for OS X, let's pack the whole app inside of a stand alone jar.

To create a standalone Java Application through Leiningen, the main class of the program has to be put into project.clj as the value of the :main key. For this class to be available during the build process, (:gen-class) has to be called within the (ns) declaration of the responsible Clojure file. Documentation on how to achieve this can be found [[http://asymmetrical-view.com/2010/06/08/building-standalone-jars-wtih-leiningen.html][here]].

#+BEGIN_SRC sh

lein compile

lein uberjar #+END_SRC sh

#+RESULTS:

When curious, try starting the jar manually

#+BEGIN_SRC sh java -jar target/clojure_desktop_app_demo-0.1.0-SNAPSHOT-standalone.jar #+END_SRC sh

#+RESULTS:

The resulting JAR file is ready to be packaged into a Mac OS X Installer.

**** By hand

This is more for testing purposes since you will still need the JRE installed.

#+BEGIN_SRC sh rm -rf /tmp/vr-restream /tmp/vr-restream.dmg mkdir /tmp/vr-restream cp target/*standalone.jar /tmp/vr-restream/vr-restream.jar ln -s /Applications /tmp/vr-restream/Applications hdiutil create -srcfolder /tmp/vr-restream /tmp/vr-restream.dmg #+END_SRC

#+RESULTS: : created: /tmp/vr-restream.dmg

**** Using the /javapackager/ tool

The following script is based on [[http://centerkey.com/mac/java/][this]] tutorial.

#+BEGIN_SRC sh cd target rm -rf *iconset rm -rf package deploy rm vr-restream.dmg mkdir vr-restream.iconset sips -z 128 128 ../resources/img/logo.png --out vr-restream.iconset/icon_128x128.png iconutil --convert icns vr-restream.iconset mkdir -p package/macosx cp vr-restream.icns package/macosx jdk=$(/usr/libexec/java_home)$jdk/bin/javapackager -version $jdk/bin/javapackager -deploy -native dmg
-srcfiles vr_logorrhoe-0.1.0-SNAPSHOT-standalone.jar -appclass vr_logorrhoe.core -name vr-restream
-outdir deploy -outfile vr-restream -v #+END_SRC

Note: If something needs to be packed into the .dmg file, this would be a way. In this example a config value is changed inside a file: For that, let's mount the .dmg file in RW mode, use sed to change the config file value and then create a new .dmg file that's again RO. This process has been inspired by: http://www.macenterprise.org/articles/creativewaysofusingshadowfiles

#+BEGIN_SRC sh

Step 1: Eject the volume that has been mounted by javapackager

hdiutil detach /Volumes/vr-restream

Step 2: Attach the read-only image with a shadow option

hdiutil attach -owners on deploy/bundles/vr-restream-1.0.dmg -shadow

Step 3: Mutate inside the image what needs mutating

sed -i '' -e "s/app.classpath=/app.classpath=vr_logorrhoe-0.1.0-SNAPSHOT-standalone.jar/g" /Volumes/vr-restream/vr-restream.app/Contents/Java/vr-restream.cfg

Step 4: Detach the currently attached image

hdiutil detach /Volumes/vr-restream

Step 5: Convert the image back to read-only, in the process creating a new image

hdiutil convert -format UDZO -o vr-restream.dmg deploy/bundles/vr-restream-1.0.dmg -shadow

#+END_SRC

After a good while (there's code signing going on), you will find a dmg file sitting in the deploy/bundles directory.

** License

Copyright © 2016 Voice Republic Media AG

*** Bundled software

This project bundles the [[http://lame.sourceforge.net/][LAME Encoder]]. LAME is under the
LGPL. Since it has been bundled in binary form and has not been
modified, it is allowed to distribute it within this project
without releasing the source of the project itself.