Wednesday, February 24, 2016

Count lines in source files

Following up on the last article, here's how to count the total number of lines your source files:

find **/src/* -regex ".*\.\(ts\|java\|less\)" -print0 | xargs -0 wc -l | tail -1

ZSH, which I'm using supports **. I believe bash cannot do that. ** would be equivalent to */*, */*/*, etc.
What this find command does after finding this filenames is to print them to standard output, but instead of separating them with a new line character, it uses NUL or \0 (the null character).

xargs then reads this. We also tell it that the filenames are separated by NUL (with the option -0), and then we give the command to be executed on each file, the wc -l command. wc stands for word count, but actually it's also capable of counting characters and lines. When we only want lines, we can use the -l argument (l as in lucky). As we are not interested to see how many lines each of the files contains, we "select" only the last line of the output with tail, because that last line contains the total.

Here is an alternative way to do the same thing:

find **/src/* -regex ".*\.\(ts\|java\|less\)" -exec wc -l {} \; | awk '{print $1;}' | paste -s -d+ | bc

In this case we find built-in feature to execute a command on each file it finds, using with the -exec argument. It is described in the manual of find. Basically, everything until the semi-colon (which as all characters that would otherwise be interpreted by the shell instead of being sent to the command, needs to be escaped). {} is replaced by the filename (the path is relative to the current directory).


wc prints the number of lines, a space, and then the filename. We use awk to print the first "word" it finds (by default, awk supposes "words" are separated by one or more spaces). I am not going to explain awk syntax, but it is a very useful tool. In fact we could have used it to do the rest of the job.

paste is a handy tool too, yet most people have never heard of it and try to replicate its features with complicated shell scripts... In this example it is going to transform this:

1
2
3

into this:

1+2+3

bc, which I believe stands for basic calculator will be used to compute the sum.


Now, there are tools to define what are "lines of code" and how to count them with more context that only some text. But, let's just ignore blank lines:

find **/src/* -regex ".*\.\(ts\|java\|less\)" -printf "sed '/^$/d' %p | wc -l\n" | sh | awk '{print $1;}' | paste -s -d+ | bc

sed, with these arguments, truncates empty lines (the regex describes "a line that begins and ends with nothing in between").
Unfortunately find does not support pipes ( | ) in the -exec argument, so we use a little trick.
That trick is to print commands to standard output, and then to interpret them with the shell, as if these commands were part of a shell script.

Find files by extension with the terminal

Today I was trying to find source files in a development project. These files have the extensions .java, .ts and .less.
I realized I never needed to do such a search before, and here's how I did it:

find . -regex ".*\.\(ts\|java\|less\)"

Let me know if you find something better.

Sunday, October 11, 2015

Populating a PostgreSQL table / database

The other day I had to populate a table with hundreds of millions of records.
Here's how I managed to do it quite rapidly:

  1. Remove all the constraints and indexes from the table.
  2. Prepare your data in a CSV file that you put on the DB server.
  3. Open a console on the DB server: psql -d myDb
  4. And here comes the star of the show, the COPY statement:
    COPY myTable (theFirstColumn, theSecondColumn) FROM '/path/to/some/file.csv' WITH CSV;

    Append QUOTE AS '"' DELIMITER AS ',' ESCAPE AS '\' if required.
    The table definition is not required if the columns in your file match their order in the table, but I would recommend against it.
  5. Add the constraints and indexes

If you really have to use INSERT statements, at least put them in one big transaction.

Note it is not going to check against NOT NULL!

Wednesday, October 7, 2015

OS X 10.11 El Capital hangs when plugging a USB device in VMware Workstation 12.0

The other day I tried to "hot plug" my iPad into OS X running as a guest, which consistently resulted in the guest OS freezing / hanging with no other solution than reboot it.

I haven't found a real solution for this, nor do I really care to investigate the problem. My workaround is to connect it before OS X starts, and this way it's properly recognized.

A quick example about geofencing (sorry, "region monitoring") on iOS with Swift is coming up!

Monday, March 16, 2015

One more hour of sleep every night

Cheerful Sunset
(no copyright information)


It's 7 pm and the sky has this nice orange - red color.
But my computer screen does not. It's "shouting" a blue-ish tone at me. But not for long!

Everybody knows that watching screens (TV, computer or handheld devices) before going to bed is bad for your sleep. What you might not know, is that if you do use screens at night, you'll sleep more and better if the screen is yellow - red instead of blue. Yes, blue! I won't start the blue/black gold/white dress war again, but it's true and you don't realize it: there is no such thing as white in nature and color analysis is one of these very complex tasks your brain "computes" yet is subjective at the end because your brain is not wired the same way than mine, so at some point you might see a gold dress when I see a blue dress. So believe me, if you are looking at your screen right now, it's blue. If you think car headlights are white, then put one car with regular headlights and one with Xenon lamps next to each other. You will probably say one is yellow-ish and the other is blue-ish.
But that's the beauty of it. If your screen goes very slowly and smoothly from blue to yellow - red, you won't notice it.

So, about that one hour of sleep. There was a study I cannot find anymore, where they had people lying in bed doing nothing, and they measured how fast these people would fell asleep. Then they would repeat the experiment with a more yellow / red light. They found people would fell asleep faster with the second setup.

Redshift on Windows

I haven't tried it, but there is an experimental version for Windows. It does seem like f.lux is older but maybe it works better, I don't know. Tell me if you find out.

Installing and configuring Redshift on Kubuntu (or any Linux distribution really)

In this tutorial I'll assume you live in western Switzerland, and that you use Kubuntu.
The procedure is similar on other systems. I am also assuming you are not changing timezones all the time. (But there is a solution for you if you find yourself in this situation, look at the documentation.)

  1. Install Redshift:
    sudo apt-get install redshift
  2. Create and open the configuration file with your favorite editor:
    nano ~/.config/redshift.conf
  3. Paste this (Ctrl + Shift + V in Konsole) :
    [redshift]
    transition=1
    location-provider=manual
    adjustment-method=randr
    [manual]
    lat=46.7
    lon=7.1

    Change the last two lines with your latitude and longitude (yes, go ahead, click on this link). You can keep all digits if you want to. If your latitude reads South (Australia, New Zealand) or your longitude reads West (North America), use negative values where appropriate.

    If you can see "GNU nano" at the top left of the console window, press Ctrl+O then Ctrl+X when you are done to close the editor and save the file (or the other way around).
  4. Start "redshift" from the Terminal to check if it works. Your screen should go a bit yellow in a matter of seconds if the sun is not up. Otherwise try to mess with the latitude and longitude or your computer clock. There should be no output on the console.
  5. All well? Time to start Redshift automatically. Open the KDE menu and type "autostart". Select the entry that appears. Click "Add Program..." then type "redshift" (without quotes). Don't select anything just type "redshift" and click OK. Click OK again to close the window.
  6. Log out and in again. Your screen should be slightly yellow. It does? Congratulations. You just bought yourself one hour of sleep each night.
Now I suggest you install a similar app like Twilight on your phone.

Thursday, March 5, 2015

Control Netflix from your phone

UPDATE: Teclado Flix has been discontinued.

Check out my new Android application Teclado Flix.



It's a remote control for people watching Netflix on their PC.
TVs from 2005 to 2014 feature HDMI inputs but cannot be connected to the Internet or run apps like Smart TVs can.
So what can Netflix fans with "old" TVs do? Just plug your laptop on the TV and watch Netflix.
But it would be silly to get up to pause and resume a video, so I made a remote control app to do that.

A computer program is required. I made a very tiny one that runs on any operating system.

Sunday, March 1, 2015

Netflix on Arch Linux (or Linux in general)

UPDATE: 
You don't need an unstable version of Chrome to play Netflix streamed media anymore. Simply download the latest stable version of Chrome.
Note that Chromium (it is exactly the same as Chrome, minus features such as syncing using a Google account and proprietary plugins) does not support the widevine and thus cannot play Netflix content.

Original article:

Netflix has thrown away its Microsoft Silverlight interface and is now using HTML5 to stream its content.

Well, almost. Not surprisingly, the videos are still encrypted and DRM'd. It is done using a plugin developed by Widevine Technologies (a Google company).
Chromium is the open-source part of the Google Chrome browser (everything but the Google branding, the syncing mechanism with the Google account and a few plugins, is open source). So nowadays you'll often find the Chromium browser on Linux distributions.

The Widevine plugin is not included in Chromium for licensing reasons, yet you would expect to be able to install it anyway. There seems to be people working on that but I couldn't make it work.

What worked for me is this:

1) Install google-chrome-dev from AUR. The reason you need the unstable version is that you need a very recent version (42+) and that you need a Google-branded browser including the Widevine plugin.
yaourt google-chrome-dev
2) Launch it with the google-chrome-unstable command.
3) Open Netflix and check if it works.
4) If it doesn't work, make sure the Widevine plugin is installed and working. Type chrome://plugins and possibly chrome://components in the address bar and check it is enabled.


Also, check out this cool browser extension Flix Plus from Lifehacker. I find it very useful!

I have to say I am impressed by this company from a business and technological point of view. Did you know they release a lot of their code with an open source license? Did you know they don't manage any hardware but rely heavily on Amazon for their computing and storage needs?
There is a trend nowadays, except for a few giants like Facebook, for most companies to transition from managing their own hardware and storage to have it them hosted by platforms such as Amazon AWS, Heroku, GitHub, Bitbucket, Microsoft Azure and other online services.
In many IT companies, there's no server room anymore, except for switches and routers, as file servers and other tools are physically located outside the company in data centers.