Wednesday, March 4, 2015

Bootstrap & Syntax Highlighting

Bootstrap is probably the hottest web framework out there right now, but try using it with the extremely popular SyntaxHightlighter by Alex Gorbatchev, which I wrote about in syntax sensation. There are at least two issues:

  1. Y scrollbars appear for no reason and
  2. there is a conflict between Bootstrap's and SyntaxHighlighter's .container class.

Enter our latest contenders:

These are both extremely light and fast but offer somewhat more quality than google pretify which I also mentioned in syntax sensation. One thing I will say about both of these is that looking at the resulting DOM, highlight.js prepends hljs- to all of its classes, almost like a namespace, so that it's unlikely that there will ever be conflicts with any other plugin. highlight.js has many languages and styles while Prism has many extra plugins like line-numbers that you can add to your build from their download page. Finally both of these new syntax highlighter's conform to the <pre><code class="language-blah"></code></pre> style that evidently is the standard for putting code into HTML documents. Who knew? SyntaxHighlighter only uses <pre class="brush: blah"></pre> which is non-standard, I guess. Nit-pick much?

Check out for yourself how Bootstrap interacts with each syntax highlighter in the iframe below, or click the link to open in a new tab. The option menu on the right side of the navbar lets you choose which syntax highlighter to see. The template is Bootstrap's theme example which you can return to by clicking the brand on the left side of the navbar. Since highlight.hs comes with 49 styles, you can peruse them from the dropdown menu. Let me know if you find anything amiss anywhere. Of course you will see the extra y-scrollbar in the SyntaxHightlighter rendition.

Bootstrap & Syntax Highlighting 3-Way

Wednesday, January 21, 2015

Single Sign On from Apache in Django using Active Directory and LDAP

UPDATE: 2015-02-25 Today I nearly crapped a cow. I was testing out a custom ErrorDocument 401 directive that would redirect back to the sign in page (BTW: that's a bad idea, IE & Firefox sign on windows are modal). I clicked OK with empty username and empty password fields, and I got the dreaded Internal Sever Error HTTP/1.1 500 page. Then because the browser had cached the empty creds, I could not get back on the server. Clearing the cache and browser history had no effect. I actually thought I had broken Apache! Stack Exchange ServerFault to the rescue. The fix is to set AuthLDAPBindAuthoritative off in httpd.conf.

So you have a nice and shiny new Django application, you successfully transitioned from development to production, and now you want to add Single Sign On (SSO) so users can use the same credentials they already use somewhere else. Sounds good, how do you do it?

TL;DR

This is surprisingly easy, although there is some new syntax to learn, and you will need to get some info from your system administrator. Here are some steps for Apache-2.4 from ApacheLounge.
  1. Follow the directions in the Django documentation on Authentication using REMOTE_USER and add RemoteUserMiddleware and RemoteUserBackend to AUTHENTICATION_BACKENDS to your settings file. This will use the REMOTE_USER environment variable set by Apache when it authorizes users and use it for authentication on the Django website.
  2. Note: This will change how Django works; for example, any authorized user not in the Django Users model will have their username automatically added and set to active, but their password and the is_staff attribute will not be set.

  3. Get the URL or IP address of your Active Directory server from your system administrator. For LDAP with basic authentication, the port is usually 389, but check to make sure.
  4. Also get the "Distringuished Name" of the "search base" from your system administrator. A "Distringuished Name" is LDAP lingo for a string made up of several components, usually the "Organizational Unit (OU)" and the "Domain Components (DC)", that distinguish entries in the Active Directory.
  5. Finally ask your system administrator to set up a "binding" distinguished name and password to authorize searches of the Active Directory.
  6. Then in httpd.conf enable mod_authnz_ldap and mod_ldap.
  7. Also in httpd.conf add a Location for the URL endpoint, EG: / for the entire website, to be password protected.
  8. You must set AuthName. This will be displayed to the user when they are prompted to enter their credentials.
  9. Also must also set AuthType, AuthBasicProvider, AuthLDAPUrl and Require. Prepend ldap:// to your AD server name and append the port, base DN, scope, attribute and search filter. The port is separated by a colon (:), the base DN by a slash (/) and the other parameters by question marks (?) such as:
    ldap://host:port/basedn?attribute?scope?filter
  10. <Location />
      AuthName "Please enter your SSO credentials."
      AuthBasicProvider ldap
      AuthType basic
      AuthLDAPUrl "ldap://my.activedirectory.com:389/OU=Offices,DC=activedirectory,DC=com?sAMAccountName"
      AuthLDAPBindDN "CN=binding_account,OU=Administrators,DC=activedirectory,DC=com"
      AuthLDAPBindPassword binding_password
      AuthLDAPBindAuthoritative off
      LDAPReferrals off
      Require valid-user
    </Location>
    
  11. The "attribute" to search for in Windows Active Directory is "SAM-Account-Name" or sAMAccountName. This is the equivalent of a user name.
  12. The default "scope" is sub which means it will search the base DN and everything below it in the Active Directory. And the default "filter" is (objectClass=*) which is the equivalent of no filter.
  13. There are several options for limiting users and groups. If you set Require to valid-user then any user in the AD who can authenticate will be authorized.
  14. Set AuthLDAPBindDN and AuthLDAPBindPassword to the binding account's DN and password.
  15. It has been reported that LDAPReferrals should be set to off or you may get the following error.

    (70023)This function has not been implemented on this platform: AH01277: LDAP: Unable to add rebind cross reference entry. Out of memory?

  16. Finally, restart your Apache httpd server and test out your site.
Now when users go to your Django site, when they open the location that requires authentication they will see a pop up that asks for their credentials.

Loggout

In addition to adding authenticated users to the Django Users model, the users credentials are stored in the browser. This makes logging out akward since the user will need to close their browser to logout. There are several approaches to get Django to logout a user.
  • redirect the user to a URL with fake basic authentication prepended to the path.
  • http://log:out@example.com
  • render a template with status set to 401 which is the code for unauthorized that will clear the credentials in browser cache.
  • from django.shortcuts import render
    from django.contrib.auth import logout as auth_logout
    import logging  # import the logging library
    logger = logging.getLogger(__name__)  # Get an instance of a logger
    
    def logout(request):
        """
        Replaces ``django.contrib.auth.views.logout``.
        """
        logger.debug('user %s logging out', request.user.username)
        auth_logout(request)
        return render(request, 'index.html', status=401)
    

Using Telnet to ping AD server

A lot of sites suggest this. First you will need to enable Telnet on your Windows PC. This can be done from Uninstall a program in the Control Panel by selecting Turn Windows features on or off and checking Telnet Client. Then opening a command terminal and typing telnet followed by open my.activedirectory.com 389. Surprise! If it works you will only see the output:
Connecting to my.activedirectory.com...
If it does not work then you will see this additional output:
Could not open connection to the host, on port 389: Connect failed
Now treat yourself and try open towel.blinkenlights.nl. Use control + ] to kill the connection, then type quit to quit telnet.

Testing LDAP using Python

  • Python-LDAP
  • So to learn more about LDAP there are a couple of packages that you can use to interrogate and authenticate with and AD server using LDAP. Python-LDAP seems to be common and easy to use. It's based on OpenLDAP.
    >>> import ldap
    >>> server = ldap.initialize('ldap://my.activedirectory.com:389')
    >>> server.simple_bind('CN=bind_user,OU=Administrators,DC=activedirectory,DC=com','bind_password')  # returns 1 on success
    1
    >>> user = l.search_s('OU=Users,DC=activedirectory,DC=com',ldap.SCOPE_SUBTREE,'(&(sAMAccountName=my_username)(ObjectClass=user))',('cn','sAMAccountName','mail'))
    >>> user
    [('CN=My Name,OU=Super-Users,OU=USA,OU=California,OU=Sites,DC=activedirectory,DC=com',
      {'cn': ['My Name'],
       'sAMAccountName': ['my_username'],
       'mail': ['my_username@activedirectory.com']})]
    
  • PyAD
  • Another Python package that can use LDAP to search an active directory is PyAD which uses PyWin32 and ADSI on Windows.
  • PyWin32
  • The only decent documentation for this is Tim Golden's website.

Alternatives

  • SSPI/NTLM
  • If users will only use the Django application on a Windows PC which they already have been authorized, EG through windows logon, then using either mod_authnz_sspi or mod_authnz_ntlm to acquire those credentials from your Windows session is also an option.
  • Django Extensions and Snippets
  • There are several Django extensions and snippets that use Python-LDAP and override ModelBackend so that Django handles authorization and authentication instead of Apache.

    Some Django extensions and snippets also exist to subclass ModelBackends to use PyWin32 to use local credentials from the current windows machine for authorization and authentication from within Django.

  • SAML and OAuth
  • Sure you could do this. You can also use SSL with LDAP or Kerebos with SSPI/NTLM. But, alas, I did not research these options althought I did come across a few references.

CSS and JS

The references section loosely based on Javascript TOC robot. It could also use the counters and the ::before style pseudo-element, but since I'm using JavaScript it doesn't make sense. But here's what that looked like anyway.

Example

first reference

second reference

Example

first reference

second reference

In case it wasn't clear above the JavaScript below is not what I'm using on this page. It was for a different approach using counters which I scratched, so these examples are very contrived and don't really make sense anymore.

Friday, January 9, 2015

Questionable Quantities in MATLAB

I am proud to introduce Quantities for MATLAB. Quantities is an units and uncertainties package for MATLAB. It is inspired by Pint, a Python package for quantities.

Installation

Clone or download the Quantities package to your MATLAB folder as +Quantities.

Usage

  1. Construct a units registry, which contains all units, constants, prefixes and dimensions.
  2.     >> ureg = Quantities.unitRegistry
    
      ureg = 
    
      Map with properties:
    
            Count: 279
          KeyType: char
        ValueType: any
    
  3. Optionally pass verbosity parameter to unitRegistry to see list of units loaded.
  4.     >> ureg = Quantities.unitRegistry('v',2)
    
  5. Units and constants can be indexed from the unitRegsitry using their name or alias in parentheses or as dot-notation. The unit, constant and quantity class all subclass to double so you can perform any operation on them. Combining a double with a unit creates a quantity class object.
  6.     >> T1 = 45*ureg('celsius') % index units using parentheses or dot notation
        T1 = 
           45 ± 0 [degC];
    
        >> T2 = 123.3*ureg.degC % index units by name or by alias
        T2 = 
               123.3 ± 0 [degC];
    
        >> heat_loss = ureg.stefan_boltzmann_constant*(T1^4 - T2^4)
        heat_loss = 
               -819814 ± 0 [gram*second^-3];
    
  7. Perform operations. All units are converted to base.
  8.     >> T2.to_base
        ans = 
               396.45 ± 0 [kelvin];
    
        >> heat_loss = ureg.stefan_boltzmann_constant*(T1.to_base^4 - T2.to_base^4)
        heat_loss = 
               -819814 ± 0 [gram*second^-3];
    
  9. Add uncertainty to quantities by calling constructor. Uncertainty is propagated using 1st order linear combinations.
  10.     >> T3 = Quantities.quantity(56.2, 1.23, ureg.degC)
        T3 = 
               56.2 ± 1.23 [degC];
    
        >> heat_loss = ureg.stefan_boltzmann_constant*(T1^4 - T3^4)
        heat_loss = 
               -86228.1 ± 9966.66 [gram*second^-3];
    
MATLAB Syntax Highlighter brush by Will Schleter

Tuesday, November 25, 2014

Convert Excel row-column reference to A1 in MATLAB

I'm sure there are many of these snippets out there. Here is yet another one.
>> rc2a1(23,196)
GN23
Enjoy. Please don't use it for evil.

Monday, October 20, 2014

NumPy in virtualenv on Windows-x64 with wheels

[UPDATE 2014-11-17] I am about to eat my own words, because Carl Kleffner has provided openBLAS dynamic libraries and headers that you can use to build NumPy>=1.91. The DLL is compiled using his static mingw-w64 GCC toolchain. From his site download 2014-08-28_OpenBLAS-0.2.12dev_PR440.7z and extract it into numpy/core folder of the NumPy-1.9.1 distribution. Then follow the directions on the OpenBLAS users Google Group. A wheel is my dropbox. You can check it's config in a Python interpreter by importing NumPy and calling numpy.__config__.show(). If you have nose, you can also run its tests by calling numpy.test(). Thanks Carl and Xianyi, this is an important milestone in making powerful numerical software free and open-sourced.

I want you to repeat after me.
"You will never be able to build NumPy on Windows x64 without the Intel Compiler Suite."
OK, was that so hard? You may have briefly entertained pipe dreams of building NumPy with OpenBLAS binaries, but then reality sank in after NumPy fails to load with this sad traceback.
Traceback (most recent call last):
  File "", line 1, in 
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\__init__.py", line 170, in 
    from . import add_newdocs
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\add_newdocs.py", line 13, in 
    from numpy.lib import add_newdoc
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\lib\__init__.py", line 18, in 
    from .polynomial import *
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\lib\polynomial.py", line 19, in 
    from numpy.linalg import eigvals, lstsq, inv
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\linalg\__init__.py", line 51, in 
    from .linalg import *
  File "c:\.virtualenvs\openblas\lib\site-packages\numpy\linalg\linalg.py", line 29, in 
    from numpy.linalg import lapack_lite, _umath_linalg
ImportError: DLL load failed: The specified procedure could not be found
Everyone said, "use dependency walker," which you did, and it told you that ABI incompatibilities meant that NumPy had no idea how to call your libopenblas.dll functions, even though they were exported and supported by libgfortran-3.dll and libgcc_s_seh-1.dll and the right versions of msvcr90.dll which you found buried in winsxs.

You got briefly excited when you saw that Carl Kleffner is actually working on a solution to this by introducing a static GCC toolchain to compile NumPy with OpenBLAS (*), but then you had a sudden insight. Hasn't Christoph Gohlke already compiled these Python libraries using Intel's Math Kernel Library (MKL)? Why yes he has! And can't you convert a bdist_wininst to a wheel? Of course you can!
$ wheel convert numpy‑MKL‑1.8.2.win‑amd64‑py2.7.exe
Now you can just pip install the wheels in your virtualenvs.
(venv)
$ pip install -U numpy-1.8.2-cp27-none-win_amd64.whl
Get your wheels here: Thanks Christoph (and Carl and Xianyi)!

(* See UPDATE at the top of this post for more info on Carl Kleffners OpenBLAS version of NumPy.)
Fork me on GitHub Creative Commons License
poquitopicante by Mark Mikofski is licensed under a Creative Commons Attribution 3.0 Unported License.
Based on a work at http://poquitopicante.blogspot.com.
Permissions beyond the scope of this license may be available at http://poquitopicante.blogspot.com/p/darn-disclaimer-and-litigious-license.html.