Monday, October 22, 2012

Saturday, September 8, 2012

New Ubuntu System Installation - Checklist

May 2013

New Cinnemon Mint / Ubuntu System Installation - Checklist


I am switching back from OSX to ubuntu, because Apple OSX is moving towards retail customers, rather than pro customers. My full reasoning is explained here.  Long live linux ubuntu!

Here are my notes of how to install a new ubuntu system:

apt-get

  • emacs and ess (note: ess may block emacs startup if R is not yet installed)
    sudo add-apt-repository ppa:cassou/emacs ; sudo apt-get update ; sudo apt-get install emacs24 emacs24-el
  • chromium and chrome
    deb http://dl.google.com/linux/deb/ stable non-free main ; wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - && sudo apt-get update && sudo apt-get install google-chrome-stable
  • terminator; rightclick for preferences; and, after you made any changes in preferences, in the now existing  ~/.config/terminator/config
          size = 900, 600
  • okular (remove sidebars, resize)
  • mplayer
  • Gstreamer-*
    the alternative is to install medibuntu:
    wget --output-document=/etc/apt/sources.list.d/medibuntu.list http://www.medibuntu.org/sources.list.d/$(lsb_release -cs).list && sudo apt-get --quiet update && sudo apt-get --yes --quiet --allow-unauthenticated install medibuntu-keyring && sudo apt-get --quiet updateapt-get -y update && sudo apt-get -y upgradeapt-get install app-install-data-medibuntu apport-hooks-medibuntuapt-get install w64codecs libdvdcss2apt-get install libdvdnav4apt-get install libdvdread4/usr/share/doc/libdvdread4/./install-css.sh
  • openssh-server
  • sshfs; fuse-zip
  • gufw and set rules against unauthorized access (night, invalid logon attempts)
  • backintime (exclude *.iso, .gvfs)
  • unison
  • mpg321 (mpg123 to play command line beeps)
  • unity-chromium-extension: these may not work.  the gmail panel always quits
  • indicator-multiload System Load Indicator (by Michael Hofmann for load status on panel).  then start it with the Unity menu (search System Load).  alas, this one does not work  under cinnemon mint, because it complains about missing gtop libraries
  • apcupsd (for UPS) and configure it according to the USB community documentation.
  • CanoScan LiDE 100:  libsane sane-utils xsane .  maybe it requires libsane-extras, too.  Thereafter, xsane works.  strangely, simplescan does not, even following this.
  • davfs2
  • flashplugin-installer; now in 12.10?
  • MyUnity (maybe kill --- too dangerous)
  • not used much: remastersys --- allows creating a dvd
  • not used much: RecordMyDesktop
  • not using much: gnome-gmail
Do not install either texlive or R through ubuntu. Their versions are seriously obsolete. Instead, follow the instructions below.
  • remove thunderbird

Do not run MyUnity or compizconfig-settings-manager (ccms).  They hose the panel-service.  if you do this, you probably need to remove a lot of the dot (.compiz, .gnome, ??) files to get them back.DONT USE: ubuntu-tweak (though MyUnity does the same):
  add-apt-repository ppa:tualatrix/ppa ; apt-get update ; apt-get install ubuntu-tweak

For Adobe Acrobat, search for Software Sources, then enable "Other Software -> Canonical Partners". This also allows removing old repositories.  Then apt-get install acroread .  Also, acroread does not work on the command line, but the GUI starter for Acrobat works.

Beyond apt

  • vmware get vmware-player . then "bash VM*".  if you want,
       ln -s /usr/bin/vmplayer /usr/bin/vmware-player.Note: everytime the host kernel or any vmware item is updated, you need to drop to the shell and run vmware-player while su by hand.  somehow, the GUI authorizer is broken.
  • skype from skype's website.  note that ubuntu has trouble with some logitech webcams.
  • TeXLive, install from source, following http://www.tug.org/texlive/quickinstall.html.  the main program is
       install-tl-unx.tar.gz
    make sure to set path to /usr/local/texlive/2012//bin/x86_64-linux/. Next, we need to install lucida fonts.  so
        echo "Map lucida.map" >> `kpsewhich updmap.cfg` ; updmap ; texhash
  • R installation instructions are at
       http://cran.r-project.org/bin/linux/ubuntu/README
    
    Basically, include
      deb http://cran.stat.ucla.edu/bin/linux/ubuntu raring/
    
    in /etc/apt/sources.list, add the signage
      sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9
    and execute
      apt-get update; apt-get install r-base
    and immediately install libatlas!!  R is 3 times faster with it than with the standard BLAS library that ships with cinnemon mint 15 olivia.  OSX has this already set.
  • Good ssh key instructions can be found at http://www.g-loaded.eu/2005/11/10/ssh-with-keys/
    Make sure to choose good pass phrases. Basically, you need to do the following on your new client:
       client> ssh-keygen -t dsa -f ~/.ssh/id_dsa
       client> cat ~/.ssh/id_dsa.pub  | ssh name@server "cat >> ~/.ssh/authorized_keys"
       client> chmod -R 0600 ~/.ssh ; chmod u+x ~/.ssh
       client> ssh-add
    
    and add a MaxAuthTries 5 and ServerAliveInterval 60 to /etc/ssh/sshd_config.
  • in /etc/default/grub, remove the "QUIET" from GRUB_CMDLINE_LINUX_DEFAULT, and change the GRUB_TIMEOUT to 5, then run update-grub
  • change /etc/grub.d/30_os-prober to timeout 3 instead of 0 (to get a menu at boot)
  • for SSD, enhance disk checking but turn off journaling: tune2fs -c 1 /dev/sda1 ; unmounted, you should tune2fs -O ^has_journal /dev/sda1. Then add discard to the options in /etc/fstab
  • kdebugdialog (not as su, but as logged in) and then disable all debug output
  • link open to gnome-open to ~/bin/open
  • make anacron less quiet: change -q in /etc/cron.d/ ; possibly change daily exec time from 7am to 3am
  • transfer .bash* , .emacs, src/* bin/*, /usr/local/texlive/ ; also .ssh
  • google office (GWoffice):
  •   add-apt-repository ppa:tombeckmann/ppa ; apt-get update ; apt-get install gwoffice
  • Handbrake video converter:
    add-apt-repository ppa:stebbins/handbrake-snapshots
    apt-get update
    apt-get install handbrake-gtk handbrake-cli
  • not needed in cinnamon mint apt-get remove unity-lens-shopping
  • does not work: edit /etc/X11/Xsession : line 83: exec >>"$ERRFILE" 2>&1  should be exec >> /dev/null 2>&1
  • for extra security, hide sudo somewhere else
  • for the backup hard drive, use hdparm -S 12 /dev/sdb1 (60 seconds)
  • Grive --- doesn't seem to work too well:
    sudo add-apt-repository ppa:nilarimogard/webupd8
    sudo apt-get update
    sudo apt-get install grive
Music:


settings---change timeouts.  hot corner on the right, not left.

Others:
  • calibre book reader
  • reinstate cron jobs

Much More Complete

debianhelp has posted http://debianhelp.wordpress.com/2012/03/09/to-do-list-after-installing-ubuntu-12-04-lts-aka-precise-pangolin/ .  In fact, it is so complete that finding one's way around it is painful.

Panel

The following apps live on my panel: chrome, Terminal, Homefolder, Ubuntu Software Center, Settings, Rythmbox, System Updater, Workspace Switcher, VMWare, Acrobat.

emacs, tex, and R are all usually invoked from my terminal.

Missing From Linux

A lightweight clean html editor and mailer.  even this blogger up produces nasty html markup that is difficult to correct with emacs.  kompozer has the wrong colors.
figure out backintime's scheduling and retrieval
how to move the menu from the top panel line to *both* the top pane line and to each window menu
improve terminal --- different colors for different machines, c-n, c-v, c-x.  add "open" to right click
Evernote -- maybe Everpad?
how to stop windows from snapping to the top panel??
alt-ctrl-hotkey on apple keyboard -- how do I abort X server and switch to virtual terminal?
create backup bootable own drive

try teamviewer for remote control.

write a script that checks all of the above and does whatever is still missing.

printer installation -- wonderfully easy.  alas, Ricoh copier printers do not work.  on ubuntu, the first print job worked fine to my lexmark usb printer; the next one produced garbage.

scanner installation -- xsane works for canon, but not the easier ones

Problem for Non-Linux Users


for some reason, 12.04.1 booted into a text login, instead of the gui login.  this persisted over 2 reboots and then disappeared.  go figure.  this would be a serious problem for a non-unix expert.  I don't know why.

there was also a problem where it would sometimes not automatically continue on restart, which was a problem for an unattended server.

Thursday, August 9, 2012

Where Apple OSX is Really Headed



Many pundits have opined where OSX is headed.  They are all wrong.

Here is my prediction: By 2018, Apple will stop supporting OSX for non-Apple app programmers, and require that all apps be purchased through their Appstore.  OSX will become like iOS in this respect, too, as it has in other respects.

Frankly, this will make sense for 99% of Apple's retail clientele.  It will prevent lots of malware and reduce support costs.  And it will make Apple money, because Appstore purchases earn Apple money.

Why wouldn't Apple focus on building certified Apple app programmers and abandon non-Apple-app programmers?  Maybe 1 in 100 buyers buy Apple hardware to program their computers for themselves, so the sale losses to programmers who would switch away to other hardware will be small. (Unfortunately, I am in the 1% minority.)  WIth enough of a retail base to reach, OSX and iOS app developers will stay and be certified.  To write a native OSX computer program will require a revocable developer id unlock code and take place in a trusted environment.  (The certified Apple developers will like some aspects of such an ecosystem, because it will cut down on free competition.  They won't like sharing revenues at the Appstore.)  But ultimately, it does not matter what software developers like.  When the Apple Mac base is large enough and the Appstore cut is bearable, they will stay.

Ultimately only two aspects will matter to Apple: making the retail clientele happy and increasing Apple's revenues.  And going the "certified developer only" route is a win-win for both.

My guess is that some non-certified programming will be allowed in interpreted sandboxed environments, at least for a while.  This will cover the needs of students learning how to program.  In this respect, it will be different from iOS, where all programming environments are outlawed. 

One alternative would be a locked OSX default mode that can be unlocked by end users.  of course, once software can unlock the computer mode, so can malware---and apple revenues will be lower, because developers could escape the Appstore.  This end-user switchable OSX will probably be an interim step.

I believe that this future direction of Apple is almost inevitable once Apple becomes a dominant power in high-end personal computer sales and developers can't escape.  The only thing that can stop it is a slowing of their momentum in PC sales, which could force them to take PC competition more seriously.  But if the Apple momentum continues, then if Apple will not go the lock-down route, they will be making a huge business mistake.

We nerds are all in trouble.

Hardware is in trouble.  And, this time, it's not even an attack by the copyright kingdoms.  Its just that there is nowhere to go.  Where can we go?  Sun is gone.  Windows is going towards trusted computing, so we will also soon no longer be able to use Windows hardware (esp. hardware from the big vendors, like Dell and HP, although all they do these days is try to ape Apple, and not very well).  There is no great hardware vendor for high-quality open software desktops.  (This is why we nerds also buy Mac hardware.)

Software is in trouble, too.  Ironically, it is exactly OSX that has pretty much killed linux on the desktop.  We have to preserve our expertise---linux on the desktop---tying the projects over until we (the nerds) will be forced to depart OSX.  We should never underestimate the future, but we may not have anywhere obvious to go.  

Richard Stallman is right to fear Apple more than Microsoft.  I do, too.


Update: http://arstechnica.com/apple/2012/09/two-months-later-developers-mostly-positive-about-os-xs-gatekeeper/  --- it's coming quickly.

PS: Here are some minor squawks about OSX: the file system could more robust.  There is no ability to lock a cpu core to a virtual machine.  Preview.app has been going backward IMHO after Snow Leopard; it is not storing its page view settings properly.  There is no package manager...well, this is what the appstore will become.  OSX feels slow relative to linux.

Tuesday, April 3, 2012

R Annoyances and Gripes

R is a superb data analysis and graphing program. It has a solid programming environment. It has great docs, although I wish it had even more "see also" and more "examples" in the man pages. many CRAN packages are great. the R team members and folks answering r-help posts are saints, though some of them can be quite grumpy.

[I try to integrate some of the comments from below.  thx everyone.]

Alas, because R is so good, I am probably too tempted to write too many programs in R. Unfortunately, for a generic programming language, R still lacks some sugar.

  1. It should be possible for a user to set an option that forces aborts immediately upon read access into a non-existing list item or data frame column (i.e., if this data frame or list that has not yet been initialized). instead, R simply silently yields NULL. Try it!
    .GlobalEnv$misspelled
    
    or even
    .GlobalEnv$misspelled[100] 
    
    do not cause a program abort. good luck finding the error later.  Gabor also suggests using the .GlobalEnv[["misspelled"]] construct, which does check.  Still, the '$' syntax is often so much nicer....
  2. It should be possible for a user to turn automatic recycling off. I have programmed bugs which I could have caught earlier if the program had bombed on non-conformable assignments. why do we need BY DEFAULT  the following to work?
    v←1:4
    w←1:8
    z←v+w
    
    Really? Are you sure it is what you wanted? If it really is, you should set an option first, or better yet, rep(v( yourself.  there should be an option(recycle="default"|"scalars-only"|"never").  you can even leave the current default, but allow *me* to turn this off.  over time, packages may switch their model, and, by default, everything should still just work without changes.
  3. An error should always clearly identify the lines in a program where it occurred.
    Simple Fix: Turn on
      options(error=recover)
    again and run my program again to understand where my program bombed.
  4. An assert()should be built into the R language---assert should be like a stopifnot() function, but with a cat message function. So, I could write
    assert( ncol(x)==ncol(y), "Sorry, but ncol(x)=",ncol(x)," != ncol(y)=", ncol(y), "\n")
    the big advantage of building assert into the language, rather than having it user defined, is that an assert error will bomb showing the correct line, not an error inside the assert function, requiring a stack backtrack.  it could even drop you into the right stack-frame at that time---an advantage currently only for the main control logic but not logic inside functions is the immediate ability to examine the variables that were associated with the failure.

    Programming by contract would be even better. (package testthat has some testing functions, but it's not the same.)   Perhaps a novel pretty syntax, like

    [ ncol(x) != ncol(y) => "Sorry, but", ncol(x), "is not", ncol(y) ]
    

    would be even better, abusing both the '[' when it starts a statement and and '=>' . Such readable and SHORT syntax, when paired with a clear error on which line the error occurred when it triggers, would encourage everyone to use a lot of assertions, including at each function start and end.
  5. there is no way for an end user (not library writer) to add his own function doc to the set of docs that one can interrogate with '?' request for doc about a function.  suggestion: use package skeleton for building full packages.  nothing for end users, though.
  6. similar---can we please add something akin to the perl6 pod to R?  please adopt some good features from perl, even though R is of course not perl.  PS: and where/what are the standard filename conventions? I think it is .Rdata for data files. right? (Is it .Rh for R inclusion files?)
  7. nice but unnecessary: Syntactic sugar: would it not be great if '$' inside a string would do a paste-with-autocollapse? How ugly is it to call my function as
    f( paste("msg is '",m,"'", collapse="") )
    
    compared to the more readable
    f("msg is '$m'")
    
    ?  but the simpler paste0 goes a long way...
  8. There should be an easier way to assign from a list to multiple objects
    (a,b) ← f()
    
    Gabor G created a great R function that facilitates this, but something like it should be built in.
  9. R should be smart enough to understand one special case to allow quickly assigning to individual elements in a data frame. Believe it or not, but
      bigframe$a[12]←12
    
    will actually copy the bigframe object a few times, instead of just replacing the single cell's content. if bigframe is big, you can count the number of assignments per second on an Intel core 2012 system. yikes! of course, I know this now, but all novices probably learn this the hard way. they may get the impression that R is slow when they (like I did before I knew) write,
      for (i in 1:10000) bigframe$soln[i]← uniroot(myf, c(-Inf,Inf), i)
    
    But
      for (i in 1:10000) soln[i]← uniroot(myf, c(-Inf,Inf), i);
      bigframe$soln←soln; rm(soln)
    
    is reasonably fast. (The data.table R package is one way to get around this, but again, this is not what novices would know.)  This is important to fix IMHO.
  10. nice but unnecessary.  I would love to have many of my functions defined at the end after my control logic at the start of my R code. thus, I would love it if I could instruct R to scan the source file for functions first before executing---as it is in perl. This behavior could be enabled by a user and not be the default.
  11. probably a bad idea on my part.  there should be "static" (local persistent) variables in R. Use very sparingly, of course.  I would use it for in-function caching of previously seen/handled cases.

Some more minor gripes:
  1. load() should have an option to write to stderr what objects are being loaded. in fact, such stderr output should probably be the default to remind the user, just as library loading usually does.  load() should also have an option to check its environment first to see if the object already exists, and load it only when still needed...sort of like a library() invocation.
  2. there should be a legacy option, without which demoted features cause warnings and errors...and let's demote attach and detach.
  3. if
      option(na.rm=TRUE)
    
    then mean(c(1:5,NA)); cor(c(1:5,NA), c(rnorm(6)) should give values. I have read about na.action, etc, but I could never figure out how to make it work.  lm() ignores NA even silently without any option.  why one and not the other?  of course, it doesn't have to be fully consistent from day 1.  but let's just get it started.  mean, sd, var, median, summary could be fixed.
  4. bad idea on my part.  paste0 is better. There should be
    option(pastechar=" ")
    option(catchar=" ")
    
    to allow setting of the default sep for paste and cat. Incidentally, I like space to be the default to separate the elements in a vector, but I do not like space before and after separate arguments to paste and cat. I can add the latter myself more easily.
  5. The commonly used descriptive summary() for a data object should also include the number of observations, the number of NA's, and the sd as built in; possibly even the T-stat.  I have programmed my own summary(), but this would be a good idea for everyone.
  6. the parallel core library is superb. finally an easy way for me to use the 8 cores in my Mac Pro! however, mclapply does not make it easy to figure out how many times the function was called. will it take another 100 days, or another 100 seconds to finish? tough to guess. a progress counter would help.  Workaround:
    sleep1sec ← function(i) {
       counter<←counter+1;
       cat("Counter=",counter, " i=",i, "\n");
       Sys.sleep(1); rnorm(1) };
    mclapply(1:1000, sleep1sec) 
    
    but unfortunately each process has its own global counter. still better than no process indicators whatsoever, though.
  7. the two graphics plotting systems should really be replaced by one. I also don't understand the difference between S3 and S4. do we need this?
  8. very minor: sleep() should be a function that sleeps, not a data object that contains how much students slept. Sys.sleep() is what I need. ok, this is a very trivial gripe. but the docs for sleep (?sleep) should say "see also Sys.sleep".
  9. why do I need both source and load ? couldn't these functions detect whether I am reading an .R or an .Rdata file and invoke the right one? both are R objects.
  10. ok.  I need to use the apply family more.  Does anyone have an idea how to abbreviate the common
    mylist← rnorm(5); for (si in 1:length(mylist)) { value← mylist[si]; ...
    
    into something that reads quicker and still has a counter?  In R, having a counter is more important than it is in perl, because iterative list creation via push is discouraged relative to list creation via indexed assignment (for speed reasons, I think).  Maybe a new iterator, like
    iterate(si, value, rnorm(5)) { ...
    
    perhaps? I know this is wild syntactic sugar, but then I write user programs and not computer languages, and I would like my programs to be readable and elegant, more than I would like R to make perfect sense.
  11. why don't more options allow mnemonics? for example, options(warn=1) means what? couldn't it be options(warn="immediately"); or, why not text(...,pos="left"), instead of test(...,pos=2)?
  12. why do some functions wrap quotes around variables? For example, why is it library(something), instead of library("something")? something without quotes should be a variable. same thing for select in subset statements. tell me: what is
    d <- data.frame( a=1:3, b=5:7 )
      a <- "b" ; a2 <- "b"
      subset( d, TRUE, select=c(a) )
      subset( d, TRUE, select=c(a2) )
    ok, if it were always optional to omit the quotes, I would understand it. but it isn't the case with a subset-select, in which I want to delete a variable.  subset(d, TRUE, select= -c(x)) works, but subset(d, TRUE, select= -c("x")) does not.  huh?
  13. merge should also be capable of merging by rowname, not just by columns.
  14. Core Team: Make your life easier!  Would it be possibly to pseudo-wikify the docs? I.e., allow collaborative suggestions? I often think "this ?... should have included a see also to ...", but there is no easy way for me to fix it right there and then for future users. the existing R bug+suggestion system is ok for bigger issues, of course, but painful for these small items. I also wonder whether my suggestions are actually welcome or a distraction. and, of course, I understand that any short edits should require package maintainer approval in the end. my guess is that many suggestions to the docs would be for the better...and, unlike code itself, docs are something that ordinary users can contribute to. after all, it is they who often need and use the docs the most.
Please don't take these as criticisms of the work that the R team has put into R. It is easier for users to gripe than it is to implement features. And obviously I am not willing or able to put these features in myself.

Of course, R has so much magic and so many features that it may well be the case that many of my gripes above already have solutions, but I just don't know them. :-( if you see any, let me know, please. And let me know what gripes should I have complained about that I missed, too.


PS: Next, I need to figure out how to create LaTeX output of regressions and data systematically. there are a few packages on cran, but I am not yet sure which one I want to use.

PPS: I just discovered Doug Bates' blog on Julia. Interesting. I don't miss what Julia provides (e.g., the JIT) too much. the stuff above bothers me more.