Upgrade to Nextcloud 14 and add OnlyOffice editor

The upgrade to Nextcloud 14 from 13 was really easy. Just delete the container, recreate the instance with the same command as before and go through the migration of the database.

In my case I ran the following statement:

docker run -d -p 8102:80 --name=nextcloud \
-v /etc/localtime:/etc/localtime:ro \
-v /volume1/docker/nextcloud/apps/:/var/www/html/custom_apps/ \
-v /volume1/docker/nextcloud/config/:/var/www/html/config/ \
-v /volume1/docker/nextcloud/data/:/var/www/html/data/ \
--link nextcloud-mysql:mysql nextcloud:latest

Post upgrading of the container I check for error messages and I received one saying that I needed to apply new indexes on two tables. I ran the occ command as suggested.

docker exec -it --user www-data nextcloud bash
php /var/www/html/occ db:add-missing-indices

Adding OnlyOffice as an in-browser editor to Nextcloud is really easy as it is well documented. I needed to do just one minor modification to the config file that OnlyOffice is providing to terminate SSL for the Document Server of OnlyOffice.

First, lets install the Document Server in a seperate Docker container.

docker run -i -t -d --name onlyoffice -p 8103:80 --restart=always \
-v /volume1/docker/onlyoffice/logs:/var/log/onlyoffice  \
-v /volume1/docker/onlyoffice/data:/var/www/onlyoffice/Data  \
-v /volume1/docker/onlyoffice/lib:/var/lib/onlyoffice \
-v /volume1/docker/onlyoffice/db:/var/lib/postgresql  onlyoffice/documentserver

Next, configure HAProxy for SSL termination and for using a virtual path (e.g. https://myserver.com/onlyoffice/). I have Nextcloud and the Document Server running on the same server.

If you are not running HAProxy as a reverse proxy, OnlyOffice have a number of pre-defined configurations for other proxies here: https://github.com/ONLYOFFICE/document-server-proxy

I have in the frontend defined to redirect /onlyoffice/ paths to the backend definition as follows:

acl is_onlyoffice path_beg /onlyoffice
use_backend onlyoffice if is_onlyoffice

backend onlyoffice
acl existing-x-forwarded-host req.hdr(X-Forwarded-Host) -m found
acl existing-x-forwarded-proto req.hdr(X-Forwarded-Proto) -m found
http-request add-header X-Forwarded-Host %[req.hdr(Host)]/onlyoffice unless existing-x-forwarded-host
http-request add-header X-Forwarded-Proto https unless existing-x-forwarded-proto
reqrep ^([^\ :]*)\ /onlyoffice/(.*)     \1\ /\2
server onlyoffice-documentserver 192.168.2.244:8103

The above code will fix the SSL termination and make sure that Document Server internal references are using HTTPS and it will change the internal routing based on the virtual path.

Lastly, install the OnlyOffice app in Nextcloud and configure it in the administration panel to use the route as you have defined.

Screen Shot 2018-09-20 at 14.59.07

Advertisements

Nextcloud 13 behind HA Proxy with letsencrypt – all in Docker containers

I want to have an Nextcloud server for my family and friends and I want to have it behind a reversed proxy so that I’ll get SSL termination and the reversed proxy can in addition serve other http-based services that I later want to expose externally or only internally.

Tasks:

  1. Setup bradjonesllc/docker-haproxy-letsencrypt docker container
  2. Setup Nextcloud 13 docker container
  3. Configure HA Proxy and Nextcloud

1. Setup the HA Proxy container

I’m already using the duckdns.org service for dynamically updating my IP-address to a domain name and I want a frontend reverse proxy for SSL traffic to Nextcloud. I.e. for routing https://mydomain.duckdns.org/nextcloud/ to the Nexcloud 13 container that I will install and to have a valid SSL certificate that is generated by the Letsencrypt service.

First I’ve setup a routing in my switch (Ubiquiti EdgeRouter X) of port 80 and 443 to the ports that I plan to expose from the reverse proxy docker container:

  • External port 80 maps to port 9980 of the HA proxy container
  • External port 443 (SSL) maps to port 9981 of the HA proxy container

Next is to create a couple of external folders, to the container, where I want to keep the haproxy.cfg configuration file and the certificates that are being generated.

  • /volume1/docker/letsencrypt/
  • /volyme1/docker/haproxy/

To work the configuration file for HA Proxy I’m copying the haproxy.cfg file from the repo: https://github.com/BradJonesLLC/docker-haproxy-letsencrypt (I could have created a copy from the container but then I don’t have to start up one in order to do so).

Copy the haproxy.cfg file to the /volume1/docker/haproxy/ folder.

Now it is time to start the container…

docker run -t -e CERTS=mydomain.duckdns.org -e EMAIL=my@mail.com -v /volume1/docker/letsencrypt/:/etc/letsencrypt -v /volume1/docker/haproxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg -p 9980:80 -p 9981:443 --name haproxy bradjonesllc/docker-haproxy-letsencrypt

If everything is starting up fine – certificates are created as they should – let’s move on to the next step of setting up Nextcloud. Please note that you must have

2. Setup the Nextcloud 13 container

I’ll be using a separate MySQL db and will link that to the Nextcloud container to avoid using SQLlite that comes with.

docker run --name nextcloud-mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql:latest

Again, create the necessary folders to keep the most important data out of the container when it is time to upgrade.

  • /volume1/docker/nextcloud/apps
  • /volume1/docker/nextcloud/config
  • /volume1/docker/nextcloud/data

Start the docker container and wait until you can reach the logon page on port 8102. E.g. http://192.168.2.10:8102

docker run -d -v /volume1/docker/nextcloud/apps/:/var/www/html/custom_apps/ -v /volume1/docker/nextcloud/config/:/var/www/html/config/ -v /volume1/docker/nextcloud/data/:/var/www/html/data/ -p 8102:80 --link nextcloud-mysql:mysql --name mynextcloud nextcloud:latest

When you can reach the webpage, follow the wizard and use MySQL as a datasource with hostname “mysql” as shown below.

Screen Shot 2018-05-03 at 11.25.00

Finish the setup wizard.

3. Final touches

Now we need to fix so that HA Proxy and Nextcloud works together and Nextcloud is accessible externally with a proper SSL certificate.

Open /volyme1/docker/haproxy/haproxy.cfg and below is my configuration file that will redirect to SSL, include SSL termination to backend services and forward any requests from /nextcloud/ to http://192.168.2.10:8102 – my Nextcloud docker container.

global
 maxconn 256
 lua-load /usr/local/etc/haproxy/acme-http01-webroot.lua
 chroot /jail
 ssl-default-bind-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
 ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
 ssl-default-server-ciphers ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256
 ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tls-tickets
 tune.ssl.default-dh-param 4096

defaults
 mode http
 timeout connect 5000ms
 timeout client 50000ms
 timeout server 50000ms
 option forwardfor
 option http-server-close

frontend http
 bind *:80
 mode http
 acl url_acme_http01 path_beg /.well-known/acme-challenge/
 http-request use-service lua.acme-http01 if METH_GET url_acme_http01
 redirect scheme https code 301 if !{ ssl_fc }

 default_backend www-backend

frontend ft_ssl_vip
 bind *:443 ssl crt /usr/local/etc/haproxy/certs/ no-sslv3 no-tls-tickets no-tlsv10 no-tlsv11
 rspadd Strict-Transport-Security:\ max-age=15768000

#Adjust paths
 acl missing_nc_path path /nextcloud
 http-request redirect location /nextcloud/ if missing_nc_path

# App definitions
acl is_nc path_beg /nextcloud
 use_backend nextcloud if is_nc

backend www-backend
 redirect scheme https code 301 if !{ ssl_fc }

backend nextcloud
 reqrep ^([^\ :]*)\ /nc/(.*) \1\ /\2
 reqadd X-Script-Name:\ /nextcloud
 option httpclose
 option forwardfor
 server node1 192.168.2.10:8102

Restart the HA Proxy container.

Last but not least, we need to update the config.php file of Nextcloud.

Open /docker/nextcloud/config/config.php file in an editor and add/replace the following values:

 'trusted_domains'=>
array (
 0 => '192.168.2.10:8102',
 1 => 'mydomain.duckdns.org',
 ),
 'trusted_proxies' =>
 array (
 0 => '192.168.2.10',
 ),
 'overwrite.cli.url' => 'https://mydomain.duckdns.org/nextcloud',
 'overwriteprotocol' => 'https',
 'overwritehost' => 'mydomain.duckdns.org',
 'overwritewebroot' => '/nextcloud',

I hope that I have covered it all in my translation of the process into this blog entry..  The server should now be accessible on https://mydomain.duckdns.org/nextcloud/

EDIT: Incorrect cipher setting in haproxy.cfg

Post my installation I tested with the Nextcloud app to reach my server and I received an SSL Initialization error. After googling the error this appears to be a common issue but after validating with the Mozilla SSL Configuration Generator I now have a working configuration and I’ve updated the haproxy.cfg file above.

Stitching an Dji Spark sphere and fix it for use in Facebook or Google Photos

Ok, a ton of guides already exists and I do not plan to do another one but I want to leave you with my experiences as I do not want to use the Dji Go 4 app for stitching the sphere – I work with the SD full image files.

The tool that I use for stitching is Hugin which is open source. I bet there are easier and better alternatives but as I just do it more for fun I won’t pay a $100 for a license. Hugin can be downloaded here http://hugin.sourceforge.net/ and is available for many operating systems.

This is the guide that I used to familiarize myself  with the Hugin software

  1. I did not create a Stereographic projection but I used an Equirectangular output from Hugin
  2. I exported it as an JPG file
  3. Post export I imported it into GIMP to make sure that it follows the required 2:1 dimension and consequently resized the canvas by modifying the height to be half the width.  Expand at the top of the image
  4. Use an EXIF editor to modify the adjusted JPG. I’m using an online editor https://www.thexifer.net/ to do this
  5. Make sure that the following values are adjusted
    1. CroppedAreaImageHeightPixels = new full height
    2. CroppedAreaImageWidthPixels = new full width
    3. CroppedAreaLeftPixels = 0
    4. CroppedAreaTopPixels = 0
    5. FullPanoHeightPixels = new full height
    6. FullPanoWidthPixels = new full width
    7. ProjectionType = equirectangular
    8. UsePanoramaViewer = true

Now you are done and ready to upload it to Facebook or Google Photos to be shared with your friends.

Note: I had some challenges to update the fields in the EXIF online tool but it worked if I first cleared the file of the EXIF metadata prior to inputing the new values.

I’m showing an example in the post below. Click image to view it in full screen.

Custom Plex DVR setup with Dreambox DM500HD

Ok, I love my Plex server. I’ve been using it almost daily since 2012. But there’s been one feature missing since I turned off my Windows Media Center server many years ago and that has been to subscribe and record shows that I follow. And when the Plex team announced DVR support in Plex I’ve been meaning to look into it.
As I’m using a satellite dish to get my channels I looked for an alternative way from the official supported devices and stumbled upon a Dreambox thread in one of the forums. Here is what I’ve done if it happens to be of interest to someone else.

Screen Shot 2017-10-25 at 22.03.03-min

  1. I bought a Dreambox 500HD S2 (satellite tuner) on eBay
  2. As it had the original firmware on it, I first needed to flash it with the “second stage loader” in order to be able to add a custom image on it. It is described at the bottom of this page: http://sources.dreamboxupdate.com/
  3. I needed to find a firmware that supported softcams as well as it supported the PlexDVR API and after a lot of Googling I found out that OpenATV should work great
  4. I downloaded and flashed OpenATV v6.1 from this site: http://images.mynonpublic.com/openatv/6.1/index.php?open=dm500hd
  5. Next step was to enable softcams and after many tries and reflashes, I managed to fine a way for it
    1. Download softcam-feed-universal_2.0_all.ipk
    2. Use FTP to upload it to /tmp (see note below – I’ve been using DCC)
    3. Do NOT use the automated install through the Plugin Manager – Open a Terminal window (DCC)
    4. Run the command: “opkg install /tmp/softcam-feed-universal_2.0_all.ipk”
    5. Now, press the blue remote button and navigate to install plugins and softcams should be listed as an option. If it doesn’t exit all the menus and try again.
  6. I installed CCcam 2.3 and started the softcam under the Softcam Panel
    Screen Shot 2018-03-28 at 23.04.39
  7. Using FTP goto /usr/keys to find a template to be used for the softcam. Modify it according to your specs and save it as CCcam.cfg in the same directory. I also updated the attributes to 755 (I don’t know if that is required)
  8. NOW – it is time to install the PlexDVR API plugin. Note that the plugin has recently been renamed to HR-Tuner Proxy
  9. Post installation follow the guide as described here: http://www.world-of-satellite.com/showthread.php?56936-PlexDVR-Setup
    Screen Shot 2018-03-28 at 23.22.34
  10. It works great for recording shows or movies and the new content will be stored into my existing libraries. LiveTV is not working perfectly – sometimes it works and some times not – but recording of shows has not failed once.

Other information:

  • I’ve been using Dreambox Control Center for Enigma2 (DCC) to upload and run commands on my Dreambox device: https://www.bernyr.de/dcce2/
  • Post setup I had issues with teletext flashing when watching LiveTV through Plex. This was easily resolved by changing the Plex server default subtitles settings from “Always on” to “Manual”

Issues:

  • I’m still struggling with getting the correct DVB substream with the correct subtitles saved in the file when doing a recording. Really annoying.

Run Nextcloud through a reverse proxy – HAProxy – with a different webroot

I wanted to setup HAProxy as an reverse proxy towards my nextCloud 12 server and I really struggled to find proper information on how to do that. As I have a number of backend services I needed a different webroot to define the request and I finally succeeded and I want to share my configuration settings.

Nextcloud is now accessable from https://myserver.se/nc/

/etc/haproxy/haproxy.cfg

global
        maxconn 4096
        user haproxy
        group haproxy
        daemon
        log 127.0.0.1 local0 debug

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        retries 3
        option  redispatch
        option  http-server-close
        option  forwardfor
        timeout connect 5000
        timeout client  50000
        timeout server  50000

frontend www-http
        bind *:80
        mode http
        reqadd X-Forwarded-Proto:\ http

        default_backend www-backend

backend www-backend
        #All requests should be in SSL-mode. SSL is terminated in HAProxy
        #and uses HTTP in backend requests
        redirect scheme https code 301 if !{ ssl_fc }

frontend www-https
        #My server certificate
        #Here's a great instruction on how to setup
        # LetsEncrypt with HAProxy https://skarlso.github.io/2017/02/15/how-to-https-with-hugo-letsencrypt-haproxy/
        bind *:443 ssl crt /etc/haproxy/certs/myserver.pem
        mode http
        option forwardfor
        option http-server-close
        option http-pretend-keepalive

        #Only allow some services to be available internally
        acl network_allowed src 192.168.2.0/24
        acl restricted_page path_beg /internal
        block if restricted_page !network_allowed

        # App definitions
        acl is_nc path_beg /nc
        use_backend nextcloud if is_nc

backend nextcloud
        reqrep ^([^\ :]*)\ /nc/(.*)  \1\ /\2
        reqadd X-Script-Name:\ /nc
        option httpclose
        option forwardfor
        server node1 192.168.2.212:80

And for nextCloud I updated the PHP configuration settings with my domain name “myserver.se” and the HA Proxy IP address “192.168.2.196” as explained here https://docs.nextcloud.com/server/12/admin_manual/configuration_server/reverse_proxy_configuration.html

 

/var/www/nextcloud/config/config.php:

  ...
  'trusted_domains' =>
  array (
    0 => 'localhost',
    1 => '192.168.2.212',
    2 => '192.168.2.196',
    3 => 'myserver.se',
  ),
  'trusted_proxies' => ['192.168.2.196'],
  'overwritehost' => 'myserver.se',
  'overwritewebroot' => '/nc',
  'overwritecondaddr' => '^192\.168\.2\.196$',

Figure out the esp8266 ADC Maximum voltage

In my quest to replace my current setup of Arduinos to esp8266 to gather sensor data I’ve been struggling with getting a similar output when using an LDR (light dependant resistor). I’ve been struggling with it for quite some time and I’ve finally come to a solution.

The issue being that the output graph from the LDR with an Arduino differs quite a bit from an esp8266 with an LDR.

After browsing for more details regarding the analog pin (ADC) on the esp8266 I realized that it does not measure between 0-3.3v but from 0-1v. When you do an analogRead on the input it outputs a value between 0-1024. But after I had introduced a voltage divider to get to 1v input I was still not getting the expected results.

In order to troubleshoot I measured the output from the ADC (i.e. digitalWrite(A0,1) ) and I received 0.6v. I thought that maybe it will output the expected max input value (but actually it was half the value).

const byte LDRpin=A0;

void setup() {
 pinMode(LDRpin, OUTPUT);
 delay(250);
 digitalWrite(LDRpin,1); //or analogWrite(LDRpin,255);
}

void loop() {
}

As a second step I wrote a small code and instead of a voltage divider I used a potentiometer to trim the input voltage to the ADC pin and the code did output 1024 when I reached 1.2v.

const byte LDRpin=A0;
volatile unsigned int ldr;

void setup() {
 Serial.begin(115200);
 pinMode(LDRpin, INPUT);
}

void loop() {
 ldr = analogRead(LDRpin);
 Serial.println(ldr);
 delay(5000);
}

As a summary the maximum voltage for my esp8266 is 0-1.2v. I don’t know if this is a common value for the esp8266 or if it was unique only for my esp8266-12e but at least there is a fairly simple way to find out.

So instead of using a 100Ω (R2) and 220Ω (R1) resistors to get to 1.0v I used 100Ω (R2) and 180Ω (R1) in the voltage divider to bring down 3.3v to 1.2v as input to the LDR. R1 was calculated using this online tool http://www.raltron.com/cust/tools/voltage_divider.asp

voltage-divider