ownCloud Planet

ownCloud
ownCloud Release Channels
July 2, 2015

ownCloud 8.1 coming soon image roundWith millions of ownCloud users, there are a wide variety of needs and requirements. ownCloud Server 8.1 will introduce Release Channels to give our users more flexibility in choosing an ownCloud version to meet their needs. Read on to find out what they offer and how to use them.

Release Channels

Some users want the latest ownCloud in order to benefit from the new features, performance improvements, and apps. Others depend on apps, which require a specific ownCloud release, or delay upgrading to a major release for other reasons. Even other users want to help ownCloud get better– checking if a bug is fixed or testing to see if the upcoming release works for their requirements– and would like a super recent development version. To offer the “best fitting” ownCloud Release to our users, we decided to set up release channels for the ownCloud software. Users can select the best installation source based on their own demands.

You pick a release channel either from the Update app in the Administrator settings of your ownCloud instance, or by choosing a package repository of a specific stable stream. You can watch the video below or read on to hear about the release channels we offer.

These are the channels we are providing:

Production

This channel delivers the latest fully tested release of ownCloud.

That does not imply that these releases are bug free. The issues are known and can be worked around. The downside of this channel, however, is that the releases are behind in terms of features and performance.

Get It

In the updater app, this channel only delivers versions that have proven their stability in the Release channel for quite some time– one or more months after the release of a major new version.

For Linux package users, administrators can decide what release they want to be on by picking a version-specific repository. They will only receive minor bug fixes and security updates until the release is no longer maintained. This allows you to choose when to upgrade to a major release yourself, but also puts the responsibility on your shoulders of making sure that you do not run an unsupported release!

If at all possible, we suggest to upgrade sooner rather than later. Doing a test on a copy of your production environment is a better way of ensuring the new version works for you than waiting and hoping all issues are found and fixed – that way, you lose out on many of the benefits of the new release, not only in terms of features and user interface enhancements but also when it comes to performance improvements and scalability.

You can find the version-specific repositories and links to packages on the Release Channels page (which is empty until the release on July the 7th).

release channels screenshot
Choosing release channels in the updater app

Stable

The latest stable release– as it was released by the ownCloud community. This channel delivers the latest released features after we have done our internal testing as well as public Beta and RC tests and fixed all nasty new bugs found.

Get It

This is what you get by default from owncloud.org/install. If you installed ownCloud from there, you will be offered an upgrade to ownCloud 8.1 and future releases on their release date.

Beta

This channel delivers dedicated test versions such as betas and release candidates. Betas and release candidates are the versions of choice for testers in preparation for an ownCloud release.

If you want to make sure that the upcoming ownCloud release will function well in your setup, grab a beta, do a test upgrade, and run.

Get It

Join the tes tpilots to get links to the latest test versions.

obs-logo

Daily

ownCloud daily builds. It contains an automatically updated source of the current development branch. This version is not recommended for production use but is mostly helpful to reproduce bugs and follow the development for the next major version.

Get It

You can find the daily package repository and archives in the testing section of the install page.

ownCloud Server 8.1 will bring many improvements, not only to the software itself, but also around it. Besides the new Release Channels, the release will introduce improved app handling and more.

read more



ownCloud
ownCloud, CERN and Smashbox
July 2, 2015

cernbox
A few years ago, CERN choose ownCloud as a component to build an innovative sync and share service at the LHC scale. After integrating ownCloud into their architecture they needed a deeper insight into the functioning of the sync client, which led to the development of the Smashbox automated testing tool. Today ownCloud has started to adopt this versatile tool for functional testing. In this post we will discuss the origins of Smashbox with CERN IT engineer and Smashbox developer Jakub ‘Kuba’ Moscicki and how Smashbox is being used in ownCloud with some members of the ownCloud Development and QA team. Next week, you’ll hear more about how Smashbox works and how you can get involved.

Origins of Smashbox

ownCloud has been used at CERN for two years and its usage is continually increasing. Due to the challenging requirements of the researchers and the infrastructure to reliably deal with the massive amounts of data coming from the scientific devices in place at CERN, the ownCloud in use has been modified in several places. The architecture of the sync and share service at CERN, CERNBox, fuses the synchronization and web access layer provided by ownCloud with the peta-byte storage system, EOS, used to handle LHC data.

Operating a service at this scale and complexity requires advanced Quality Assurance tools in order to verify operational state of the service as well as manage service updates. This is especially true for software running on a user system as problems there leave little room for IT to help the user, unlike on the servers where backups and other recovery mechanisms are in place. Moreover, at CERN scale even obscure corner cases become common and problems require time-consuming direct interaction between IT personnel and users. And on top of that, the multi-platform nature of the client and the lack of control service providers have over the user systems further increases the number of possible versions and variations of clients interacting with the servers.
cms-event
The idea started when the CERN team had developed some scripts with test cases that exercised the syncing. When a new configuration was implemented, the scripts were run. More interesting cases were added, a name was invented and automated Smashbox testing was born.

Smashbox at CERN

Smashbox testing became a routine before any new sync clients were added to the environment. The checks are described by Kuba as ‘very mean.’ The team tries to come up with scenarios which are difficult to handle. Tests are run continuously and in varying conditions. For example, hard to reproduce issues can depend on a timing dependent problem which only happens under very heavy load on the server, requiring many hours of running the tests under a high, simulated load in order to trigger the problem often enough so they can gather data and the ownCloud Development team can track the issue down and fix it.

Smashbox and ownCloud

ownClouders first got wind of the existence of Smashbox when it was demoed at a meeting at CERN. Klaas Freitag from the client team was particularly interested, as it allowed easy testing of the API used by the client to talk to the server. Next, the CERN Smashbox repo got forked and several ownCloud developers and testers got working by writing tests and adding support for new API’s. It is currently in the public ‘smashbox’ repository on github and there is ongoing work to merge our changes back into the upstream Smashbox project. As most of the work is in the tests, rather than the framework itself, this shouldn’t be a big problem, though the entanglement of tests and framework in the current repository would probably benefit from separation.

Kuba is glad that ownCloud has picked up Smashbox. As he points out, CERN has a culture of sharing and collaborating.

Next week we will dive a little deeper into how Smashbox works, what it is being used for, and where it will go in ownCloud.

Thanks to CERN and Jakub Moscicki for their time to answer our questions

read more



ownCloud
8.1 is Nearly Here, Spread the Word
July 1, 2015

ownCloud 8.1 coming soon image round
We’re about a week away from the release of ownCloud Server 8.1! There is much that is new and improved in this release and we need your help to make sure everybody knows.

Social Media

Much of the conversation around ownCloud happens over social media. And it is your advocacy that will help others find out about the release! You can re-tweet and share the messages we put out on our Facebook, Diaspora, Twitter and Google Plus accounts and join in on the discussion about ownCloud Server 8.1 with hashtag #owncloud on Twitter, Google+, Diaspora and Facebook!

News Sites

A very important area where we could us your help is with submissions to news sites. That includes the likes of Slashdot, reddit and many others you can think of, especially those in your local language– they should have a post about the ownCloud release and you can make that happen!

Here is a nice text you can use, or translate, in order to submit the ownCloud Server 8.1 announcement to sites:

This release marks significant under the hood improvements, increasing scalability and performance of syncing and file operations while making ownCloud a better platform for developers to build upon. Security enhancements, integrated documentation links and more control in the admin panel over external storage, LDAP and encryption make ownCloud more secure and easier to use.

The link to our announcement blog will be:  http://owncloud.org/blog/owncloud-8-1-raising-the-bar-on-security-and-performance
The link starts working on Tuesday the 7th, when we publish the release. Another important link, also not yet working, will be http://owncloud.org/eight-one which is where you can find an overview of all the improvements in ownCloud 8.1!

We have also created nice banners you can put up on your blog or website to promote the upcoming release! These are the sizes available:

Example code:

<a href="http://owncloud.org/features"><img src="https://owncloud.org/wp-content/themes/owncloudorgnew/assets/img/features/ownCloud-release-460x70.png" alt="ownCloud release" width="460" height="70" /></a>

Will look like this:

ownCloud release 460x70

These banners will get updated whenever a new release is coming or has been published, so you don’t need to change anything once they are up.

The upcoming ownCloud Server 8.1 matters. The release brings many improvements (that link is going live on release day!) and we’re very proud of what is coming. Of course, in preparation of the release, you should plan on joining (or organizing) a release party to meet fellow ownClouders in real life and discuss all this awesomeness.

So get out there and spread the word!

read more



Lukas Reschke
Combining ownCloud and Google calendar for public room availability
June 24, 2015

In my coworking space we are using ownCloud calendar to keep track of the availability of our conference room which we are also renting. However, we want also to be able to show publicly the room availability without disclosing personal information to the public. Even more limiting, since we use Jimdo to host our website we can’t execute any server-side code.

Currently we have a calendar shared using a service account to all administrative users which is used to track availability. Basically looking as following:

Reservations in ownCloud calendar

Unfortunately, the default ownCloud calendar is not able to share calendar publicly, there is since a longer time a Pull Request to add this functionality but unfortunately it is unlikely that this one is going to be merged anytime soon. Also we need to be able to embed the calendar easily on other web pages.

The first solution that came to my mind is Google Calendar since it allows easily embedding calendars within web pages. However, we wanted to keep the data under our own control and also leverage our existing tools.

Luckily ownCloud is providing CalDAV access to calendars and also iCal export due to the super awesome sabre libraries that we use. Sabre offers also classes to read objects such as iCal files. This means that to remove sensitive information from an calendar and get a new iCal file only a few lines of code are required:

<?php
// Use "composer require sabre/vobject" to get the required libraries
require_once('./vendor/autoload.php');

use Sabre\VObject;

// Configure your data
$remoteHost = 'https://cloud.smartworksg.ch';
$calendarName = 'sitzungszimmer';
$username = 'service';
$password = 'XXXXXX';

// Get ownCloud calendar
$curl = curl_init($remoteHost . '/remote.php/caldav/calendars/'.$username.'/'.$calendarName.'?export');
curl_setopt($curl, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($curl, CURLOPT_VERBOSE, true);
$ics = curl_exec($curl);
curl_close($curl);

// Parse calendar file
$calendar = VObject\Reader::read($ics);

// Replace all SUMMARY fields with a value of "Reserved"
foreach($calendar->children() as $children) {
    if($children instanceof VObject\Component\VEvent) {
        $children->SUMMARY = 'Reserved';
    }
}

// Put the resulting ICS to /var/www/public.ics
file_put_contents('/var/www/public.ics', $calendar->serialize());

This file can then get easily added to a cronjob and using Google’s calendar “Subscribe to public calendars” functionality the sanitized calendar can get shared to the public and embedded in web pages:

Reservations displayed publicly in Google Calendar

read more



Klaas Freitag
ownCloud Chunking NG
June 22, 2015

Recently Thomas and me met in person and thought about an alternative approach to bring our big file chunking to the next level. “Big file chunking” is ownClouds algorithm to upload huge files to ownCloud with clients.

This is the first of three little blog posts in which we want to present the idea and get your feedback. This is for open discussion, nothing is set in stone so far.

What is the downside of the current approach? Well, the current algorithm needs a lot of distributed knowledge between server and client to work: The naming scheme of the part files, semi secret headers, implicit knowledge. In addition to that, due to the character of the algorithm the server code is too much spread over the whole code base which makes maintaining difficult.

This situation could be improved with the following approach.

To handle chunked uploads, there will be a new WebDAV route, called remote.php/uploads.
All uploads of files larger than the chunk size will go through this route.

In a nutshell, an upload of a big file will happen as parts to a directory under that new route. The client creates it through the new route. This initiates a new upload. If the directory could be created successfully, the client starts to upload chunks of the original file into that directory. The sequence of the chunks is set by the names of the chunk files created in the directory. Once all chunks are uploaded, the client submits a MOVE request the renames the chunk upload directory to the target file.

Here is a pseudo code description of the sequence:

1. Client creates an upload directory with a self choosen name (ideally a numeric upload id):

MKCOL remote.php/uploads/upload-id

2. Client sends a chunk:

PUT remote.php/uploads/upload-id/chunk-id

3. Client repeats 2. until all chunks have successfully been uploaded
4. Client finalizes the upload:

MOVE remote.php/uploads/upload-id /path/to/target-file

5. The MOVE sends the ETag that is supposed to be overwritten in the request header to server. Server returns new ETag and FileID as reply headers of the MOVE.

During the upload, client can retrieve the current state of the upload by a PROPFIND request on the upload directory. The result will be a listing of all chunks that are already available on the server with metadata such as mtime, checksum and size.

If the server decides to remove an upload, ie. because it hasn’t been active for a time, it is free to remove the entire upload directory and return status 404 if a client tries to upload to. Also, a client is allowed to remove the entire upload directory to cancel an upload.

An upload is finalized by the MOVE request. Note that it’s a MOVE of a directory on a single file. This operation is not supported in normal file systems, but we think in this case, it has a nice well descriptive meaning. A MOVE is known as an atomic and fast operation, and that way it should be implemented by the server.

Also note that only with the final MOVE the upload operation is associated with the final destination file. We think that this approach already is a great improvement, because there is always a clear state of the upload with no secret knowledge hidden in the process.

In the next blog I will discuss an extension to this that adds more features to the process.

What do you think so far? Your feedback is appreciated, best on the ownCloud devel mailinglist!


read more