Code By
Lucas Mouilleron

An Open Source selection

sparkBoilerplate

Last update : 23/11/2015 - 16:57

A simple Spark REST boilerplate.

Screenshot

Install

  • Install JDK 8
  • Install Apache Ant
  • Install Apache Ivy
  • Get dependencies : an getDependencies

Run and test

  • Run : ant run
  • https://localhost:8087/hello

Authentication

  • JWT
  • Generate keys :
    • Private key : openssl genrsa -out private_key.pem 2048
    • Private key java format : openssl pkcs8 -topk8 -inform PEM -outform DER -in private_key.pem -out private_key.der -nocrypt
    • Public key java format : openssl rsa -in private_key.pem -pubout -outform DER -out public_key.der
  • Get token : curl -k https://localhost:8080/login/ilovekate
  • Acces restricted area : curl -k --header "token: eyJhbGciOiJSUzI1NiJ9.eyJuYW1lIjoidXNlcm5hbWUiLCJkYXRlIjoxNDQzNzgwODU3NDcyfQ.A6b6Hg1QyYgYUV8J9wff6SvclX90Ydmx6sd8OzTAXYJ6gLpkXMBaHvOLtyxu35hSiiVwrOljnfLSg__tlbzh6PG8KrMuezwCQHttnQzowfp5CxNWM5mEXcMjHiCMLGW3X_p5MV6hm7pe7M8aBDKlZj__OHEMPogFGSga5HhnRDnRfzY49cW9CgzdtfCY3c-wWwabyoy75kQTk5GG2KUOZPy5xKT9EJvL1JLlGKkCl4Il8zNGm2cpP68_hIqCohLqEfbMXjBdccYU7DsjQowBxtjQcZD92pu-6rHKhWcJVlNC32BqvPQXn5laKCm9Dpq703Km_IWvEPz0LPXDtIZ9Gg" https://localhost:8080/protected

SSL

  • Generate keystore :
    • keytool -genkey -keyalg RSA -alias sparkBoilerplate -keystore keystore -validity 3600 -keysize 2048
    • "What is your first and last name?" has to be the domain name
  • Self signing : keytool -export -alias sparkBoilerplate -file certificate.crt -keystore keystore
  • CA signing :
    • Generate CSR (certificate request) : keytool -keystore keystore -certreq -alias sparkBoilerplate -keyalg rsa -file certificate.csr
    • Get from CA the certificate.crt file
  • Generate trustore : keytool -import -file certificate.crt -alias sparkBoilerplate -keystore truststore
  • Test keystore : keytool -list -v -keystore keystore
  • In this boilerplate, password is password and host is localhost

pythonBoilerplate

Last update : 23/11/2015 - 15:53

Python for stats boilerplate.

Screenshot

Installation

  • https://github.com/lucasmouilleron/macInstaller/blob/master/commands/python

iPython

  • ipython create profile lucas
    • %pylab
    • %load_ext autoreload
    • %autoreload 2
  • in shell rc, alias ipy=ipython --profile lucas

Miscs

  • bayesianMethodsForHackers :
    • ipython notebook
    • then navigate to the books

picoBoilerplate

Last update : 23/11/2015 - 15:49

picoCMS is a lightweight flat file CMS.

Screenshot

Features

  • A nice picoCMS web boiler plate
  • Stack : picoCMS, twig, scss, requirejs, bootstrap, fontawesome
  • Build : install, build, watch, composer, grunt, bower

Install requirements

  • Install NodeJS : http://nodejs.org/download
  • sudo npm install bower -g
  • sudo npm install grunt -g
  • sudo gem install sass
  • sudo gem install --pre sass-css-importer
  • Install composer :
    • curl -sS https://getcomposer.org/installer | php
    • mv composer.phar /usr/local/bin/composer

Install

  • curl -sS https://getcomposer.org/installer | php && composer install (or php composer.phar install)
  • npm install

Build

  • Edit the package.json file
  • grunt build to build everything
  • grunt watch:scripts while coding

Browserify, CommonJS, dependencies

  • Browserify is used to compile the app
  • If module is not CommonJS, use browserify-shim (for shimming and deps) and browser (for aliases) in package.json
  • If module is not available via bower or npm, use napa

Pico plugins

  • myMetas :
    • adds custom metas in contents
    • metas defined in config.php
  • myNavigation :
    • adds custom navigation to templates
    • users meta status wehter to display or not a page (draft is hidden, default status is published)
    • uses meta order (number)
  • myTags :
    • adds tags to content
    • tags.html template must be defined to list all tags (use {% for tag, tagPages in tags %})
    • tag.html template must be defined to list all pages of a tag (use {% for page in pages %} and {{ meta.currentTag }})
  • myPlugin :
    • example of a plugin running only on a template (on before_render, test $template)

Admin

  • ftp and edit content files

Credits

Thanks to guys at http://picocms.org for their great CMS

thirdPartyServices

Last update : 23/11/2015 - 15:46

(Non :)) exhaustive guidelines, tips and tests for (more or less) popular third party services.
Includes : Facebook, Twitter, Instagram, Youtube, and more.

Motivations

Guidelines, tips and tests for (more or less) popular third parties services.
The goal is to help integrate these services in digital campaigns or services.

Guidelines and tips

Gudelines, good pratices and tips we gathered wandering.

Tests

Some tests have been implemented in web/tests.
Use the navigation to explore them.

Build

  • npm install && grunt build
  • JS : web/assets/js/main.js
  • CSS : web/assets/css/scss/main.scss

browserifyBoilerplate

Last update : 23/11/2015 - 15:45

Simple browserify boilerplate.

Features

  • A nice frontend browserify boilerplate
  • Stack : browserify, compass, bootstrap, fontawesome
  • Build : install, build, watch, grunt, npm, bower

Install requirements

  • Install NodeJS : http://nodejs.org/download
  • sudo npm install bower -g
  • sudo npm install grunt -g
  • sudo gem install sass
  • sudo gem install --pre sass-css-importer

Build

  • Edit the package.json file
  • grunt build to build everything
  • grunt watch:scripts while coding

Browserify, CommonJS, dependencies

  • Browserify is used to compile the app
  • If module is not CommonJS, use browserify-shim (for shimming and deps) and browser (for aliases) in package.json
  • If module is not available via bower or npm, use napa

TODO

  • bower and browsershim

twitter2Emulator

Last update : 23/11/2015 - 15:43

Screenshot

Setup

  • Download an emulator (snes9X)
  • Download some ROMs
  • cd server
  • npm install

Run

  • Start the emulator and a ROM
  • node server/app.js
  • Go with to clientController/index.html with your browser for additionnal controlling and logging

angularBrowserifyBoilerplate

Last update : 23/11/2015 - 15:36

An Angular Browserify boilerplate.
Features a simple sample app.

Screenshot

Features

  • Our Angular Browserify boilerplate.
  • Stack : browserify, angular, scss, bootstrap
  • Build : grunt, bower, npm, browserify, install, build, watch

Install

  • Install NodeJS : http://nodejs.org/download
  • sudo npm install bower -g
  • sudo npm install grunt -g
  • sudo gem install sass
  • sudo gem install --pre sass-css-importer

Build

  • Edit the package.json and package.local.json file
  • grunt build to build everything
  • grunt watch:scripts while coding

Angular, Browserify, CommonJS, dependencies

  • Browserify is used to compile the app
  • If module is not CommonJS, use browserify-shim (for shimming and deps) and browser (for aliases) in package.json
  • If module is not available via bower or npm, use napa

Notes

ionicBoilerplate

Last update : 23/11/2015 - 15:34

An Ionic framework boilerplate.

Screenshot

Features

  • Stack : scss, ionic, angular
  • Build : install, build, watch, grunt, bower

Install requirements

  • Install NodeJS : http://nodejs.org/download
  • sudo npm install -g cordova
  • sudo npm install -g ionic
  • sudo npm install bower -g
  • sudo npm install grunt -g
  • sudo gem install sass
  • sudo gem install --pre sass-css-importer

Install

  • cd _build && npm install
  • platforms :
    • ionic platform add ios
    • ionic platform add android

Run

  • cd _build && grunt build
  • Simulator : ionic emulate ios
  • Build : ionic build ios
  • Browser : open in browser www/index.html

authMethods

Last update : 23/11/2015 - 15:32

Client < > Server authentication and authentication transfer methods in the web context.

Screenshot

Authentication

Authentication is the process of confirming identity.

This process is used in client / server apps to grant access to resources.

Once a resource owner is authenticated, this state of authentication needs to be persisted or transfered.
Here are some possible methods to do so in the case of stateless systems, such as REST APIs and apps over HTTP in general.

Security preambule

Hash function

A hash function is any function that can be used to map digital data of arbitrary size to digital data of fixed size, with slight differences in input data producing very big differences in output data.
The output of a hash function is called the hash.

Data integrity

A simple way to ensure a piece of data was not modified when transfered from a sender to reciever is to hash it.

This will produce the signature of the data.

The sender signs the data and sends the original data and the signature.
The reciever resigns the data and if the signature is the same, the data was not modified.

This method does not guaranty the aunthenticity of the data, as it can be forged and signed by anyone.

Digital signing

A digital signature is a mathematical scheme for demonstrating the authenticity and integrity of a digital message or document.
Digital signing involves a shared secret key, known only to the sender and reciever.

A simple way of signing data is to append the shared secret key to it and hash this concatenated data.

This will produce the secret signature of the data.

The sender secret signs the data and sends the original data and the signature.
The reciever re secret signs the data (using the shared secret key), and if the signature is the same, the data was not modified since it has been signed by the sender and was issued by someone who knows the shared secret key.

This simple method does not encrypt the data.

HTTPS

HTTPS is HTTP over TLS.
It means requests performed with HTTPS can't be wiretapped.

The content of the request (header + query parameters) and the response (headers + body) are encrypted and not sniffable.

One caveat is that HTTPS requests query parameters can be loged on web servers.
If logins and passwords are passed via GET query parameters, they can be found on web servers log files.
Using request headers or POST parameters is therefore recommanded, as they are not logged by servers.

HTTP basic authentication

Basic authentication is the simpliest form of stateless authentication method.
The user credentials (login and password for example) are sent to the server within the request headers.

This method over HTTP is obviously unsafe as the credentials could be obtained by sniffing the HTTP requests.
This method is safe only when used over HTTPS.

Flow

+--------------------+                                       +--------------------+
| CLIENT | | SERVER |
| | | |
| 1. Sends the | GET /user | |
| credentials with | Authorization: Basic czZCaGRSa | |
| every request | ------------------------------------> | 2. Validates the |
| | HTTP 200 OK | credentials and |
| | {name:"lucas"} | process request |
| | <------------------------------------ | |
+--------------------+ +--------------------+
  1. The user sends its credentials with every request, within the request headers (the credential string is base64(login:password))
  2. The server validates the user credentials, processes the request and sends back the data.

Pros

  • Very simple
  • Secure over HTTPS

Cons

  • Requires HTTPS
  • Does not allow authentication delegation (such as OAuth)
  • Does not allow expiry
  • The crednetials must be sent with every authenticated call to the resource
  • The credentials validation is performed on every call

Resources

Cookie and server persisted sessions

Cookie and server persisted sessions is the most widely used authentication persistence method.
This method involves a client side cookie and a server side stored session.

A server side stored session is an object containing relevant user data. It is stored on the server.
When the user calls server methods, the server retrieves the user's object from its store (and may use it as a context).

Implementation examples : PHP sessions, ASP sessions or a CMS custom sesssions system.

PHP sessions flow

+--------------------+                                       +--------------------+
| CLIENT | | SERVER |
| | | |
| 1. Logs in | POST /login | |
| | username=...&password=... | |
| | ------------------------------------> | 2. Validates the |
| | HTTP 200 OK | user against db |
| | Set Cookie PHPSESSIONID | and create the |
| 3. Stores the | <------------------------------------ | session |
| cookie | | |
| | | |
| 4. Sends the | GET /user | |
| cookie with every | Cookie PHPSESSIONID | |
| request | ------------------------------------> | 5. Finds the |
| | HTTP 200 OK | session in the |
| | {name:"lucas"} | sesssion store and |
| | <------------------------------------ | process request |
+--------------------+ +--------------------+
  1. The user logs in, for example by providing his credentials
  2. The server validates the user credentials and fetches some relevant data (such as the login, the cart, etc.) and stores them to a session object (persisted as a file on the server)
  3. A cookie is installed on the client referencing the server side `object id
  4. For every next call, the session cookie is sent
  5. The server reads the object id from the cookie, retrieves relevent data from the session file, processes the request (using the session data as a context, in this case the login) and sends back the data

Pros

  • Quite easy to implement
  • The browser automatically handles sending the cookie to the server on every request
  • Battle tested

Cons

  • Not easy to use accross multiple server side languages (eg a nodejs and a php server)
  • Not stateless. The sessions are stored on the server. Not ideal when scaling up.
  • Cookies don't do well across different domains

Resources

Stateless tokens

A stateless token is a self defined amount of secret signed data.

Generaly, it is composed of a header, a payload (the data) and the signature.
As the payload is not encrypted, it should not contain senstive informations, such as the password of the user. It should contain its login though.

In a client / server context, the token issuer is the server.
The client carries the token and sends it with every server request, as a mean of authentication.

Because the token is secret signed by the server, the token can not be forged by a third party.
Because the token contains the payload, the server does not need to store any data on its side.

For the token not to be sniffed and reused as is, the use of HTTPS is recommanded.

An example of implementation is Json Web Token (JWT). They are url-safe (which means the header and payload are base64 encoded) and are therefore very convenient for client / server web apps.

JWT flow

+--------------------+                                       +--------------------+
| CLIENT | | SERVER |
| | | |
| 1. Logs in | POST /login | |
| | username=...&password=... | |
| | ------------------------------------> | 2. Validates the |
| | HTTP 200 OK | user against db |
| | {token:"..."} | and create the |
| 3. Stores the | <------------------------------------ | token |
| token in a cookie | | |
| | | |
| 4. Sends the | GET /user | |
| token with every | token="..." | |
| request | ------------------------------------> | 5. Validates the |
| | HTTP 200 OK | token and |
| | {name:"lucas"} | processes request |
| | <------------------------------------ | |
+--------------------+ +--------------------+
  1. The user logs in, for example by providing his credentials
  2. The server validates the user credentials and fetches some relevant data (such as the login, etc.) and add them into a token. The token is sent back to the user.
  3. The token is stored in cookie on the client for further usage
  4. For every next call, the token is sent. The token can be sent as a GET parameter or within the request header.
  5. The server processes the request :
    • The server extracts the header and payload from the token and resign then with the secret key
    • If the signature is the same, the token is valid
    • The server may as well check the token is not expired. For this purpose, the expiry date can be stored in the payload.
    • The server then processes the request (using the token data as a context, in this case the login) and sends back the data

This JWT flow is actually an OAuth2 staeless implicit flow (see below).

SSO usage

JWT token can be used for SSO puropose.

It involves an authentication server and other services in need of authenticated users.

  • Ideally, the authentication server register services : configuration of the service redirect page after login, and the private shared signing key
  • Every service must know the authentication server private shared signing key
  • When the user accesses a service :
    • If a JWT token is provided (cookie, get param), the service validates the token :
      • If the token is valid, using the payload (email, name, etc.)
      • If not, redirect to the authentication server login page
    • If no JWT token is provided, redirect to the authentication server login page
  • When the user logs in on the authentication server login page :
    • The authentication server validates the user and produces a JWT token
    • The user is then redirected to the redirect page of the service with the JWT token (get param)

Pros

  • Stateless and performance, as the token is self defined and does not need to be stored server side
  • Works easily with multiple language servers (for example PHP <-> nodejs)
  • No cross site requests flaws

Cons

  • The token must be added as a parameter with every authenticated call to the resource
  • Does not support revoking (because it is stateless)
  • Requires HTTPS

Resrouces

OAuth

Oauth permits to grant a consumer (an app, for example Coca Cola) authorization on behalf of a resource owner (a user, for example you) to perform actions on a third party resource server (a 3rd party service, for example Facebook).

To do so, the consumer creates an app within the 3rd party service platform. The consummer app comes with a consumer ID and a consumer secret.

The consumer secret is never exposed to the resource owner and is used only to authentificate the client when requesting the user access token.

The consumer then retrieves the user access token on behalf of the resource owner and sends it along with every call to the resource server.

This process gives better flexibility for the authentication process and allows delegation :

  • At no point the user credentials are known to the consumer
  • A user access tokencan be revoked by the resource owner

OAuth 2.0 code flow

+--------------------+                                       +--------------------+
| RESOURCE OWNER | | SERVER |
| via its browser | | 3rd party service |
| | | |
| 2. Logs in and | ------------------------------------> | 3. Validates the |
| grant access to | | user against db |
| the consumer | | and produce code |
| | | |
| | | |
| +------------------------------------------- | |
| | | | |
| | | | |
| | | | |
| | | | |
| | | | |
+---------------|----+ | |
| | |
^ | 4. Passes the code to the consumer on the | |
| | redirect URI | |
| | | |
| 1. Redirects to 3rd party login page | |
| GET /oauth2/authorize | |
| consumerID=...&redirectURI=... | |
| | | |
| v | |
| |
+--------------------+ | |
| CONSUMER | | |
| Client app | | |
| | | |
| 5. Requests access | POST /oauth2/token | |
| token | code=...&consumerID=... | |
| | &consumerSecret=... | |
| | ------------------------------------> | 6. Validates the |
| | HTTP 200 OK | consumer against |
| | {token:"...",refreshToken="..."} | db and create the |
| 7. Stores the | <------------------------------------ | token |
| token in a cookie | | |
| | | |
| 8. Sends the | GET /user | |
| token with every | token="..." | |
| request | ------------------------------------> | 9. Validates the |
| | HTTP 200 OK | token and |
| | {name:"lucas"} | processes request |
| | <------------------------------------ | |
+--------------------+ +--------------------+
  1. The consumer redirects or sends the resource owner to the 3rd party service login page. The consumer id and the redirect URI are passed along.
  2. The resource owner logs in and grants access to the consumer (permissions may be presented for review)
  3. The 3rd party server validates the user credentials and produces a code
  4. The 3rd party server redirects the resource owner to the redirect URI with the generated code
  5. The consumer requests a user access token to the 3rd party server with the code recieved, its consumer ID and its consumer secret
  6. The 3rd party server authenticates the consumer and issues the user access token and the refresh token
  7. The token is stored in cookie on the client for further usage
  8. For every next call, the token is sent. The token can be sent as a GET parameter or within the request header.
  9. The server processes the request by validating the token and sends back the data

The consumer can refresh the user access token if it _expired or is about to expire.
The consumer calls 3rd party server refresh token endpoint with its consumer id, consumer secret and the refresh token retrieved on step 6.

Other OAuth 2.0 flows

The pin flow

The pin flow is very similar to the code flow. The difference is the step 4 is not automatic.
This flow is typically used if the consumer app is not web capable and therefore can not perform the step 4.
The code, called the pin, is given to the resource owner so he can input it manually in the consumer app.
The pin is smaller than the code so it easier for the resource owner to manipulate it.

The implicit flow

The implicit flow is a simplified code flow optimized for clients implemented in a browser using a scripting language such as JavaScript.
In the implicit flow, instead of issuing a code, the3rd party service returns the user access token directly.
In this flow, the consumer is not authentificated by the 3rd party server.

OAuth 1.0

OAuth 1.0 is the ancestor of OAuth 2.0.

Main differences are the user access token never expire, the consumer app needs to sign some requests and more roundtrips are involved.

It is still used by Twitter.

Pros

  • The consumer can't access the resource owner credentials
  • Revoking tokens

Cons

  • Not stateless
  • OAuth 1.0 is a bit painful
  • Requires HTTPS

Resources

OAuth stateless

This is a stateless implementation of OAuth 2.0.
Like OAuth 2.0, the consumer does not know the resource owner credentials.
Because it is a stateless flow, no tokens or codes are persisted, which is great for performance and scaling up.
But because it is a stateless flow, it is not possible to revoke a user access token.

Implementation :

  • The code and user access tokens are JWT tokens so the flow is stateless
  • The code provided in step 4 is data (user id, client id and expiry) signed with a secret know only to the 3rd party service (and not the consumer)
  • When the consumer requests the user access token, the 3rd party server verifies the code and makes sure the consumer ID in the code is the same as the one provided by the consumer. Like in OAuth 2.0, it still authenticates the consumer.
  • The user access token issued by the 3rd party server is a JWT token containing relevant data (login, etc.)
  • When the consumer performs calls, the token is verified (signature and expiry) by the 3rd party service
  • Because codes and user access tokens are signed by the 3rd party service, they can't be forged by consumers or owners

youtubeToMeme

Last update : 23/11/2015 - 15:29

Youtube to animated meme generator. Screenshot

Features

  • Downloads a Youtube video, converts it to an animated gif and adds a custom meme text on top of it
  • Colors, auto linebreak
  • Based on nodejs youtube-dl and gifify
  • PHP image magick gif frame annotation

Install

  • Install NodeJS : http://nodejs.org/download
  • ffmpeg : brew install ffmpeg
  • libtool : brew install libtool
  • imagemagick : brew install imagemagick --build-from-source
  • giflossy :
    • brew install automake
    • git clone git@github.com:pornel/giflossy.git && cd giflossy && autoreconf -i && ./configure && make && make install
  • imagick php :
    • sudo pecl install imagick
    • vi /etc/php.ini and add extension=imagick.so
  • cd libs/youtubeToGif && npm install

Run

  • php cmd.php A7TaY8HWYd8 71 5 "PRETTY GOOD JUNIOR" 500 BOTTOM
  • php cmd.php 6Hn8qnsucwo 30 5 "PRETTY ... PRETTY ... PRETTY ... PRETTY ... PRETTY GOOD" 500 TOP
  • php cmd.php puo1Enh9h5k 49 2 "ROMANO LE RELOU" 500 BOTTOM

TODO

  • minimal UI
  • Linux install
  • POSITION_BOTTOM flickers