Data processing basics

Data preprocessing methods are divided into the following categories; (i) Data cleaning, (ii) Data integration, (iii) Data Transformation and (iv) Data Reduction.


Efficiency in order

Image from XKCD.

I need an extension for my research project because I spent all month trying to figure out whether learning Dvorak would help me type it faster.


Dark pattern

In short, a dark pattern is a crappy user experience that intentionally makes it difficult or almost impossible without help.


Recapturing our identities: Enabling citizens to own & monetise our personal data

The big idea

The most powerful asset of the digital age is data. And data is really a new asset class, maybe bigger than other asset classes, like land under the agrarian economy, or an industrial plant, or even money. And all of you — we — create this data. We create this asset, and we leave this trail of digital crumbs behind us as we go throughout life. And these crumbs are collected into a mirror image of you, the virtual you. And the virtual you may know more about you than you do because you can’t remember what you bought a year ago, or said a year ago, or your exact location a year ago. And the virtual you is not owned by you — that’s the big problem.
So today, there are companies working to create an identity in a black box, the virtual you owned by you. And this black box moves around with you as you travel throughout the world, and it’s very, very stingy. It only gives away the shred of information that’s required to do something. A lot of transactions, the seller doesn’t even need to know who you are. They just need to know that they got paid.
And then this avatar is sweeping up all of this data and enabling you to monetize it. And this is a wonderful thing because it can also help us protect our privacy, and privacy is the foundation of a free society. Let’s get this asset that we create back under our control, where we can own our own identity and manage it responsibly.

Uncluttered your life

When your life’s too cluttered and overwhelming then you may shut down and procrastinate by lying lazily on the couch and just watching the TV or your smart phone.

When that’s the case then start uncluttering both your work hours and your private time. Two questions that have helped me to do that and to find what is most important are:

  • What would I work on if I only had 2 hours for work today?
  • If I had just 1 hour of free time today then how would I spend it?

Use these to get out of an old rut, to question your normal day a bit and to find your top priorities.

Then see what you can eliminate, minimize or perhaps delegate of the things that are not contained in your answers.


Thintelligence

The state of mind where a person does something without considering the consequences. The idea may seem brilliant at first, but the after-affects usually prove to be deadly. This phrase was invented by Michael Crichton in his book Jurassic Park (the character Malcolm says it)


Build your own image dataset from search engine

Finding or getting the right dataset is painful process. Here’s an idea : let’s scrape images from search engine.

Combination of category and variations would pull series of queries to wrest images from search engine, before download an image from an url.  Source code that scrape close to 1M of images overnight (I do not owned the code, however I think it is very useful). Really helpful to getting train sets for classifier.

The pipeline of developing dataset. Original diagram by D Grossman.

Read the details from the original post.


SketchUp : My first hour

Since the last three years, my top circles in the UK are artists and architects by genus. And of course, I am far disparate apart from them. I enjoy all of their works. Creative is one thing is that I have a strong keen and try to break free. I want to learn it.

Since the beginning, I asked few times and inform of my interest of learning 3D modelling. But I’ve been procrastinating..

 

The first start

Lately, I am learning bits of everything on arts and eventually invested (again) some interest in 3D.

5AM or so today, I shoved computer and dive. Finally I’ve installed SketchUp.

Oh 3D is mindblowing, man!

The real reason why I wanted to learn 3D is the hope to discover different types of patterns or way into my creative product. I only used the touchpad to draw the object as above. I’d say I quite familiar with tools like Sketch and Photoshop, so this is an entirely different world. Controlling in multiple axes environment is annoying, but hey – that what makes a 3D environment is 3D, right? I don’t have a mouse, so it putting more weight to the process.

After created object that not went as it should be, I thought I should learn to do real world objects, as what my architect friends do. So, I picked one thing that’s the most common and easy enough for me to make myself comfortable with control and SketchUp essential features. To simulate this, I drew a basic house.

Lets have a TV and dining table outside, why not 😊

I love arts. However, art and me is like a distant cousins. I’ve spent years to appreciate the concept and learn, but the result are always terrible and distasteful. Nonetheless, I still believe it is learnable.

Colors, typography, patterns and other bagful components.. Oh man, those things are very hard!


WordPress : Battling with the White screen of death

Updating a deaf theme (no within-panel update) should be easy job.

Just upload, renaming folder for transition, and hit refresh.

Blank screen. 😦

Nav to /wp-admin. Here is also white screen. And I had no backup!

I made mistake today. I accidentally renamed theme folder without deactivate the current theme first. The upshot? White screen of death. My friend Aqeel is on his way for an appointment for the project, possibly asking for collaboration for the website. And yet I updating the website.

The problem was, I hosted WordPress, naked, on DigitalOcean; with no cpanel, no PhpMyAdmin etc – putting more cream to challenges.

I learnt something. I also learnt there’s no clear way solution were found.

Here’s the way to solve this.

  1. Check the current_theme values on by accessing your database on wp_options. WordPress has fail-safe mechanism for this – if your current theme has problem, it will fall to another theme for that matter.
  2. Rename the intended theme as temporary name in wp-content. For example tweentyseventeen_new.
  3. This way, WordPress should load as it should because fail-safe exceptions are passed.
  4. If you’re lucky you will see your dashboard. Now change the theme. Fixed.
  5. If not working, rename plugins folder temporarily to disable all plugins. Repeat step one.

Moral of the story; do not maintain anything without backup.


Time versus word count