Showing posts with label ridiculous rants. Show all posts
Showing posts with label ridiculous rants. Show all posts

Thursday, May 14, 2015

[MATLAB] Do *not* use `obj.empty` to preallocate object array

FYI: You do not need to use `obj.empty` to preallocate an object array.

In fact as soon as you assign a value to any element in the object array it grows the array to that size, which allocates (or reallocates) RAM for the new object array, therefore defeating the point of preallocating space.

From Empty Arrays section of OOP documentation:

“If you make an assignment to a property value, MATLAB calls the SimpleClass constructor to grow the array to the require size:”

Instead if you want to preallocate space for an object array, grow the array once by assigning the last object first. This requires the class to have a no-arg constructor. Each time you grow your array you will reallocate RAM for it, wasting time and space, so do it once with the max expected size of the array. See Initialize Object Arrays and Initializing Arrays of Handle Objects in the OOP documentation.

  >> S(max_size) = MyClass(args)

Another option is to preallocate any other container like a cell array (best IMHO), structure or containers.Map and then fill in the class objects as they are created. An advantage to this is you don’t have to subclass matlab.mixin.Heterogeneous to group different classes together.

  >> S = cell(max_size); args = {1,2,3;4,5,6;7,8,9};
  >> for x = 1:size(args,1), S(x) = MyClass(args{x,:});end

The only time to use an empty object is if you want it as a default for the situation where nothing gets instantiated, and you need the it be an instance of the class. Of course any empty array will do this, IE: '', [] and {} are also empty.

  >> S = MyClass.empty
  >> if blah,S = MyClass(args);end
  >> if isa(S, 'MyClass') && isempty(S),do stuff; end

Another reason might be to clear defaults if the constructor is called recursively, although obj.delete will do the same thing.

I hope this helps someone; it definitely helped me understand the odd nature of MATLAB. This behavior is because everything in MATLAB is an array, even a scalar is a <1x1 double> read the C-API mxArray for external references and mwArray for compiled/deployed MATLAB for more info.

MATLAB = Matrix Laboratory
Class definitions didn’t appear until 2008. Other languages like C++, Java, Python and Ruby are object first. So the empty method is meant to duplicate the ability to be empty similar to other MATLAB datatypes such as double, cell, struct, etc. IMO outside of MATLAB it's a very artificial and somewhat meaningless construct.

Wednesday, April 15, 2015

Recommended Python Project Layout

[UPDATE 2018-09-04] Links to Cookiecutter and Bootstrap a Scientific Python Library from the National Synchrotron Light Source II (NLSL-II).

[UPDATE 2016-07-19] Lately I've preferred using core instead of lib for the main package modules.

[UPDATE 2015-06-04] Create top level package to bundle all sub-packages and package-data together for install.

Been looking for a good, comprehensive, credible guide:

  1. Pretty good links in this SO Q&A:
    1. What is the best project structure for a Python application?
    2. Especially this one:
      1. Open Sourcing a Python Project the Right Way
  2. And maybe, maybe theses ones:
    1. Learn Python The Hard Way Exercise 46: A Project Skeleton
    2. The Hitchhiker’s Guide to Python! Structuring Your Project by Kenneth Reitz
      1. Repository Structure and Python also by Kenneth Reitz
    3. How to Package Your Python Code: Minimal Structure by Scott Torborg
    4. Interesting Things, Largely Python and Twisted Related: Filesystem structure of a Python project by Jean-Paul Calderone
  3. Of course understanding Python Modules and Packages
    1. The Python Tutorial: 6. Modules
  4. An understanding of how to install packages, and roughly I guess how pip and setuptools interact with distutils is good
    1. Python Documentation: Installing Python Modules
  5. Way later down the line it helps to understand distutils and setuptools for deploying packages
    1. Python Packaging User Guide
    2. Setuptools
    3. Python Documentation: Distributing Python Modules
    4. How to Package Your Python Code by Scott Torborg
    5. The Hitchhiker’s Guide to Packaging
  6. There are also a packages that will create a boiler plate project layout for you but I wouldn't recommend them except as reference guides - the tutorial by NSLS-2 being the notable exception, PTAL!
    1. Bootstrap a Scientific Python Library: This is a tutorial with a template for packaging, testing, documenting, and publishing scientific Python code.
    2. Cookiecutter: A command-line utility that creates projects from cookiecutters (project templates), e.g. creating a Python package project from a Python package project template.
    3. PyPI: Python Boilerplate Template

It's hard to pin a standard style down. Here’s mine:

MyProject/ <- git repository
|
+- .gitignore <- *.pyc, IDE files, venv/, build/, dist/, doc/_build, etc.
|
+- requirements.txt <- to install into a virtualenv
|
+- setup.py <- use setuptools, include packages, extensions, scripts and data 
|
+- MANIFEST.in <- files to include in or exclude from sdist
|
+- readme.rst <- incorporate into setup.py and docs
|
+- changes.rst <- release notes, incorporate into setup.py and docs
|
+- myproject_script.py <- script to run myproject from command line, use Python
|                         argparse for command line arguments put shebang
|                         `#! /usr/bin/env python` on 1st line and end with a
|                         `if __name__ == "__main__":` section, include in
|                         setup.py scripts section for install
|
+- any_other_scripts.py <- scripts for configuration, documentation generation
|                          or downloading assets, etc., include in setup.py
|
+- venv/ <- virtual environment to run tests, validate setup.py, development
|
+- myproject/ <- top level package keeps sub-packages and package-data together
   |             for install
   |
   +- __init__.py <- contains __version__, an API by importing key modules,
   |                 classes, functions and constants, __all__ for easy import
   |
   +- docs/ <- use Sphinx to auto-generate documentation
   |
   +- tests/ <- use nose to perform unit tests
   |
   +- other_package_data/ <- images, data files, include in setup.py
   |
   +- core/ <- main source code for myproject, sometimes called `lib`
   |  |
   |  +- __init__.py <- necessary to make mypoject_lib a sub-package
   |  |
   |  +- … <- the rest of the folders and files in myproject
   |
   +- related_project/ <- a GUI library that uses myproject_lib or tools that
      |                   myproject_lib depends on that's bundled together, etc.
      |
      +- __init__.py <- necessary to make related_project a sub-package
      |
      +- … <- the rest of the folders and files in your the related project

Thursday, March 19, 2015

Python issue 22516: Administrator rights required for local installation

[UPDATE 2015-07-02] Check out Python Bootstrap a continuously integrated build of Python-2.7 for Windows that can be installed without admin rights.

This issue starts simple enough, but then unravels to reveal some very interesting insights. Evidently creating a Python for Windows installer that does not depend on administrator rights is not as easy as it seems. The question comes down to what we really need. Steve Dover from Microsoft breaks it down like this:

  1. Python as application
  2. This is when someone wants to use python to write scripts, do analysis, etc. Python is an application on their windows machine just like MATLAB or Excel. This could be installed per-user and possible without administrative rights.
  3. Python as library
  4. This version of Python could be used to embed Python in an application. Something similar to what pyexe and other python-freezing packages do. This could be a zip file.
  5. Python as-if system
  6. This version would be installed on a system, possibly by system admins, in a custom windows build, similar to the way that Python is integrated into Mac and Linux. It would require admin rights, and could be used by system admins to add custom functionality to the corporate OS.

Let me say 1st off that I think that #3 is absurd. I can't imagine Windows system admins ever using Python in this way. Most of them have never even heard of Python. And there is an entire .NET infrastructure to do exactly this. Why would you use Python instead of C#? Windows will never, ever be like Linux. It is not a POSIX environment, it does not use GNU tools and it does not need Python.

I think that #1 and #2 could serve the same purpose and should really be the only option available. Users who want to use Python on Windows should unzip the Python Windows distro to their System Root (ie: C:\Python27) and use it. No admin rights required. End of story. It should contain the msvcrt90 redistributable and all libraries it needs. There should be no other dependencies.

There is also a 0th option - do not distribute a binary for Windows at all. Let Windows users build from source themselves, or recommend alternate distribution, which Python.org already does on its alternative distribution page.. This is what Ruby does, and perhaps it's the best way to satisfy everyone. But the fact that official Python is available for windows is a very nice thing. Althoght Enthought and ActiveState have been around for a long time, they are private and could go out of business. Nevertheless, this does seem to be the path people are taking.

Anaconda, from Continuum Analytics, a relative newcomer, founded by Peter Wang and Travis Oliphant formerly from Enthought, seems to have become, almost overnight, the most popular source of Python on windows. It's baby sister, Miniconda, is less know but merely installs the conda package/environment manager and Python-2.7 which can be used to install more Python version and packages, whereas Anaconda preinstalls most sci-data packages. The only major concern for me with Anaconda is that it is closed source. Is it the python.org version built out of the box?WinPython on the other hand is open source on github and offers both 32 and 64 bit versions that do not require admin rights to instal. Enthought is also closed source and PortablePython only offers an older out of date 32bit version. There is also PythonXY but for me it seemed buggy.

Not sure what the future of Python on Windows will look like. If you are interested in shaping that future, I suggest you contact one of the Python devs and let them know what your use case is.

Thursday, April 3, 2014

My fears about MATLAB

Someone just sent me a funny email about funny MATLAB bugs. When I say "funny" I mean in the geeky-(in fact I didn't even get why they were so funny)-way. I probably was targeted for this funny email thread because I have suggested switching from MATLAB to Python, but I think my reasons for suggesting this switch have been misunderstood. It doesn't have anything to do with bugs or the quality of MATLAB software - IMHO MATLAB is outstanding. My complaint is more practical. I am afraid of the consequences of the facts that MATLAB is closed-source and TMW is privately held. I hope that this perspective is useful, however what I express below is my opinion and based on my experiences not necessarily on facts, so please forgive me if there are some inaccuracies.

Here are my fears about MATLAB enumerated, and why I suggest switching to Python:

  1. If MATLAB goes out of business, your code is useless, and they could decide to close shop any day, because they’re not publicly traded.
  2. You can’t use it without a license! In my case I have to VPN to work to use it, this makes me not want to work from home because using vpn is just a smidge slower and another hurdle to getting started and of course forget working on airplanes or anywhere where there’s no internet.
  3. The $$$ for the toolboxes is obscene! Especially considering they are repackaging what is already free in the public domain in C/C++, FORTRAN and Python. It’s a racket. Ostensibly you could argue that the money is for support, but there are similar paid support options for Python (Anaconda, Enthought, ActiveState, etc.) that are substantially cheaper, and rival MATLAB support options to varying degrees of success.
  4. MATLAB is not portable, unless of course you use the MATLAB compiler toolbox and one of its toolbox add-ons like MATLAB builder-NE, builder-JA or builder-XL. That right there floors me as a scam, you can’t port it to someone unless you buy not one but 2 extra $$$$$ toolboxes and they are by far the most expensive toolboxes! And that’s the way it is with everything they sell. EG parallel toolbox is useless without the extra distributed computing toolbox, both of which are $$$$$! And you pay per number of machines in the computing cluster. Back to portability. If you want to share your MATLAB code with someone they will have to purchase MATLAB to use it.
  5. There is not enough consideration for backwards compatibility. I believe that TMW tries to minimize backwards incompatibility, but they still do it anyway, and in their business model, where they only make money if you maintain your license, they have a strong incentive to introduce backwards incompatibility. Other languages have the opposite incentive, in order to maintain the code-base of external packages, libraries and toolboxes that would fail if backwards incompatibilities were introduced. IE you will never find backwards incompatibilities between any version of Python-2.7.x (or in between versions of Python-3.x.x, although Python-3 is not strictly backwards compatible with Python-2).
  6. It is very powerful, but it is not the most powerful coding platform out there. FORTRAN as ancient as it is, still exceeds MATLAB and the newer Julia is faster than them all! See the comparison chart where is says “High-Performance JIT Compiler”.

Other than these few issues, I revere MATLAB and have used it extensively for decades. I rarely have issues with it and it is clearly one of the best analysis tools for science and engineering, but it isn’t the only one and it’s debatable if it’s the best.

I adhere to the philosophy of "use the right tool for the right job" and that nothing is binary, even quanta come in different flavors.

Friday, March 21, 2014

Universal Scientific Prayer of Hope and Gratitude

Oh Great Universe!

I praise your infinite complexities,
your beautiful creations,
and above all your life giving ability.

[Please let our understanding of <insert whatever>
be sufficient for <insert whatever>.]

Thanks!

Friday, August 23, 2013

Jedi Nation

In Star Wars when Naboo, Leia's homeworld is destroyed by the Death Star, Obi Wan slumps over in the Millennium Falcon and utters something like, "I feel as if ten thousand souls suddenly screamed out, and then were silenced."

Hmm, is Obi Wan using his Jedi powers to snoop on the universe? One could see it that way, but the verb "snoop" carries the connotation that he did not have the snoopee's best interests at heart. And Obi Wan was all heart.

Hence the Jedi Nation.

Let us take the current climate of anti-terrorist snooping. It brings to mind images of kicked down doors in the middle of night, subjects disappearing based on flimsy wiretap evidence. But what if we take the Jedi approach? What if the next day, Obi Wan shows up to the subjects apartment, offers to buy her some coffee and says, "I got a feeling from the Force that you're feeling a but unhappy with the current government and some other stuff. You know all these wars over oil piss me off too. Want to talk to someone about it? Are you in a bad spot right now? I could help you with your rent for awhile. Are you looking for work? Let me measure your mitochondria. You're a bit old to start the training, but the force runs strong in you. We'll make a Jedi out of you yet!"

If she still resists, Obi Wan can try Jedi mind tricks. "Theses are not the innocent civilians you are looking for." If she tries to kill Obi Wan, then he might be forced to slice her up with his light-saber in self defense.

Tuesday, June 11, 2013

Religion can be philisophy, aristocracy or imaginary friend

Disclaimer: I know this is a touchy subject, and I am not the most tactful person, so please
stop reading now!
if you have strong views on religion. What I am about to express is my opinion, and is not meant to influence or offend anyone. If you are already offended then I apologize, and hope that by turning back now you can avert any more offense.

Another theory; religion could be dissected as either a philosophy or an aristocracy. Wait, hear me out. I know that is way oversimplifying something so complex and evidently intricately woven into the human condition. But I am expressly thinking about understanding the will of each religion's deity/deities - what is done with those instructions is a different topic.

So let's consider, hypothetically a religion has a deity or some deities. This essentially what defines a religion right? That it has gods? Perhaps a religion has no gods, merely guidelines that were divined by a group of humans that are now revered for their amazing insight. That sounds a bit like a philosophical cult which is my first proposition, but I'm getting ahead of myself. Now that we've established a god or gods, how does information exchange occur?

  1. There is an elite class of god listeners who alone hear god's messages and then repeat them to the rest of that god's followers.
  2. All of that god's followers attempt to divine their god's meaning and then share and debate their theories to come up with some consensus.
Number one is clearly a form of aristocracy because the god has divined who shall be the people who receive the message and make decisions for others just as a monarch is generally chosen through some cosmic means. However if the god-listener class is elected it might possibly be considered some form of democracy. More likely the god-listener class takes that right through an exertion of their power (either by force or through influence), which might make it either a tyranny or oligarchy. Perhaps these types of institutions are all called republics - a small group represents a larger group. But it can go terribly wrong if the larger group doesn't question the validity of the small group. I wonder is it a sin to question the pope? Or a pastor's interpretation of the bible. Even a simple Sunday morning comic strip has multiple interpretations; isn't it more likely that a literary work of unknown origin transcribed multiple times may have ambiguous meaning? I'm not saying that we can't ultimately come to a consensus, there is meaning everywhere that we can all agree on, e.g. that murder is generally bad is agreed by all. What I mean is that unquestioning acceptance of religious, governmental or scientific dogma is both lazy and very dangerous. As Socrates said in the Apology, "an unexamined life is not worth living."

Number two sounds like a philosophy to me. I like it.

OK, let's take this a few steps further. Now replace god or gods with some other belief. Say god = the universe? Or gods = scientific theories, because let's be frank, no matter how much proof we have, even acceptance of a law is still merely just a belief. We believe that electrons tunnel through energy barriers because we have seen so much evidence that suggests convincingly that it may be true. But Einstein and Copernicus and Galileo can attest to the fact that even "scientific" theories and laws evolve and shift as new evidence comes to light. So I digress, my point with this last exercise is that religion has many societal parallels.

I also realized, while talking with my wife about suffering and grief that even if you can't hear your deity or deities message, when you are in need, merely believing that they exist and love you, may be a solace, and I like that too. The universe is a cold hard place, and it's always nice to have a friend.

Equality increases self esteem

Similar to my post on "the second law of infodynamics" this post proposes another completely hypothetical theory of human social interaction.
Equality increases self esteem.
Right now my son loves wearing pink, and says today, "I'm wearing a dress." Next week it will be a different color. He is completely innocent. Theses issues seem trivial to us today in the socially enlightening 21st century. In fact we view it as a victory over the absurd and old-fashioned dogma about distinct gender roles.

So lets examine that dogma. What was its motivation? I propose that it was a defensive coping mechanism. To cope with what? What could possible happen if a boy did wear a dress? Or a woman was a combatant? Or a same sex marriage occurred? Did that mean that I might start wearing dresses, because secretly I wanted to but my society told me it was wrong so I felt insecure and bad about myself? If I was secure about my individuality, why would I care what another person did? Merely being irked or irritated is not a reason for outrage is it? No there has to be a deeper reason. Our prejudices are manifestations of our inward fears. We are racist because we seek to dehumanize and theretofore justify the luxuries we take for granted at the expense of others' suffering. We are sexist and homophobic for the same reasons.

But what happens when we remove these barriers? The we don't have to be defensive. There is nothing to cope with. We can feel good about ourselves whoever we are. Equality increases our self esteem.

Monday, June 3, 2013

please start your project major version at zero

This post is probably a repeat of something on Coding Horror, but in case it hasn't been stated before, I'd like to make the case for starting your version numbering at zero. Why, well, because some code is available before it has been thoroughly tested and depends on user feedback to determine when the code has enough of the bugs worked out to consider it mature. During this time its version number may be changing rapidly, but the one thing that distinguishes a fledgling package from a robust one is the zero in front of the version number. It says, hey I was just born. I might not be complete. I might have bugs. I might break or die. I'll let you all know when I get the code equivalent of a bar-mitzvah by changing my zero to a one.
if major_version:
    print "I'm mature.",
    print "This is release #%d." % major_version
else:
    print "I'm newish,",
    print "so I might still have some big issues."
Is that so hard?

Tuesday, July 3, 2012

Google redirect virus

I visited regexbuddy and regextester, then eew, every Google search leads to blank page. I think regextester was the culprit. Quick search (on my phone) yielded numerous hits - suggested solution is malware scan.

Thursday, May 24, 2012

apple in my eye

[UPDATE 2013-04-06]

I should have started this list ages ago.

Yes, you are so sleek, ouch you cost so much money, wow Facetime, but really, you can't copy an audio CD, or even a data CD for that matter, without jumping through hoops? Oh don't worry, apple's infinitely better by 50MMM in profits lossless format, which is only 50MB per song, so much smaller than the raw format, is what I should rip all of my music to so I can have like 6 songs on my, oh only works on Ipod or Ipad, Ihole. Point is, I have to go through 3 ridiculous steps to do something that should only take one step.

Number #1: Can't duplicate an audio CD.

P.S. It goes without saying that of course Ubuntu flavored Linux does this piece of cake - all I had to do was put it in my machine and Ubuntu asks, "Want to copy CD? Play it? Want a martini?" Why yes and all of the above, and yet you do this with zero profit? How so? I will love Linux til the day I die.

Brasero is the Linux app that does this, thanks to the thankless folks at GNOME.

P.P.S. Download Burn. It works. Now I'm lovin it.

Sunday, August 21, 2011

Apple drama

I can't wait for Google to start making their own hardware. Or for Android machines to start taking over a larger market share. I get Android. I don't get Apple. For all the hype, so far not a single thing has worked hitch free. Ah, but the gadgets are sexy. The macbookpro is so sleek and cool, the iPad so convenient.

Sunday, August 14, 2011

Fairs

I must be fun-challenged, because while everyone else is lapping up carny food and gawking over watercolored bears and jewelry made from scrap flatware, I'm completely miserable. I'm irritated that I've spent gobs of money on crap. To me the whole thing is a sham. The idea of carny food always sounds delicious, but the reality is usually grossly-overpriced over-glorified gut-busting greasy garbage. The bands are fine, I love music, so the free fairs have at least got that, but I haven't been wowed enough by carny music to justify paying. And pay dear. Movies at the Parkway (RIP) were cheaper! I guess the only thing I like is the people watching. And as a kid I remember liking the rides. Maybe I've got to just suck it up and pretend to have fun for the kids. But honestly I'd rather be outside, walking, not choking my arteries, relaxing, enjoying nature for free. Ah, that sounds nice.

Fork me on GitHub