Author: shantibhushan

  • False positive demystified

    It happens sometimes when your colleagues come from a non-English language talking in a common language such as English, there are phrases which don’t readily convey what you want to say.

    The other day, when a friend of mine used “It’s a false positive” so suggest something, the other colleague gave a puzzled expression “What’s a false positive?”. Up until that point, I must admit neither did I have a clear understanding, but looking at the two words – false, positive – I could muster up an intuitive explanation on the spot.

    But back in my mind, I wasn’t really sure if I did explain the meaning simply using the dictionary definitions of those two words apart. I mean, it isn’t hard. But it was one of those phrases that you want to know more about with some level of confidence & clarity. Maybe it’s just me.

    The other day, when working with some study materials for a course about machine learning, I came across a brilliant piece of explanation that finally answered the meaning of “false positive” in one fell swoop. Take a look at this table below

    Actual = CorrectActual = Incorrect
    Prediction = CorrectTrue positiveFalse positive
    Prediction = IncorrectFalse negativeTrue negative

    Does it make sense? Absolutely yes – but I’ll explain if it doesn’t.

    The prediction is a machine learning result saying something is correct or incorrect (in my example the problem the machine learning is trying to solve/predict is if a patient has cancer or not looking at the size of the tumor & patient’s age). The actual is what you know – based on evidence or tests done beforehand proving that the patient did indeed have cancer or not. Lining up correct/incorrect with what the prediction says against what actually is makes the understanding about “false positive” much more clearer to me now.

    So, if you thought something was correct but it actually isn’t then it’s a false positive.

    The other way round, if you thought something was incorrect but it turned out to be correct then it’s a false negative.

    Makes much more sense now.

  • Asahi Brewery Tour in Moriya, Ibaragi

    Asahi Brewery Tour in Moriya, Ibaragi

    We visited Asahi Ibaragi Brewery for a free factory tour today. The tour was a guided tour that left a very good impression on all of us.

    We booked our tour online from Asahi’s website. They offer options to reach their factory either using your own car or using a free bus ride from nearest station which is Moriya. If you take your own car, then you don’t get to taste the beer (don’t drink-n-drive) so your best bet is to use the free bus ride and not miss the chance to taste 3 different kinds of beer in the end of the tour.

    We took the 10 am bus from Moriya. The bus took just under 15 mins to reach the factory. It was a small mini-bus for about 30 people. Upon reaching the factory, we were warmly greeted by a tour guide from Asahi who confirmed our tour reservation and handed out a small numbered card assigned to us. The card was needed in the final beer tasting event at the end of our tour.

    Shortly thereafter, we were led into a small theater room to watch a short introduction video about Asahi. It was less of an advertising shot, but more about their emphasis on how Asahi strives to cultivate the best quality beer with a short video footage about their strict quality procedures.

    After watching the intro video, our tour guide led us towards the factory area. Along the hallway, she explained the various steps involved in making a high quality beer at Asahi. The factory is huge and the hallways are long so there’s plenty of guided posters along the walls to stop & read through the steps. It starts from cultivating wheat, barley, hops to fermentation and into filtration and distillation to final bottling. The tour-guide explained everything in plain simple terms and where we could, she showed around the factory areas from our hallway above. Because this was a weekend Sat tour, we didn’t get a chance to witness anything actual in action. Still, the experience of actually seeing it up close was remarkable.

    We did get a chance to actually eat barley and touch & smell the hops – key ingredients for a quality beer, followed by a tour of the entire beer production process from barley-mashing to packaging.

    Towards the end, we enjoyed freshly brewed beer in open-area tasting room. The entire tour lasts about 90 minutes. Just before heading back to our return tour bus, we did some gift shopping at a small shop near the entrance. I particularly chose to pick some cakes made out of beer & whiskey (yes!) and some high cacao chocolates (again to go with whiskey!). There aren’t too many souvenirs – mostly some snacks, bottles of whiskey, beer glasses, paper-clips and T-shirts with Asahi written on it.


    It was a wonderful experience with Asahi. I definitely suggest visiting the brewery with your family & kids.

    Kanpai!

  • Fukuroda Falls trip this year 2018

    We had a long awaited trip planned to Fukuroda Falls. This is one of the recommended spots to visit in Japan. Ibaraki’s famous Fukuroda Falls is located near the northern city of Daigo, roughly two hours drive from the capital of Mito (so it was about 2 hours from my place in Tsukuba). As it has been named as one of the three most beautiful waterfalls in Japan there is never a bad time to come take it in; in fact, the locals claim that you have to see it in all four seasons in order to really grasp its beauty. During the summer it is framed with thick, green foliage; in the winter it partially freezes into a delicate glass sculpture; the spring brings new flowers and increased volume from rains; but the added contrast of the red leaves in autumn against the coursing water definitely increases its popularity by quite a margin.

    See some of our photos on this trip. If you are interested in some of the simple mountain climbing courses, please check them out here, I’ve got some photos for all seasons in this PDF too.

    Definitely recommend to visit these falls once, and yes it’s definitely a family trip.

  • Mount Tsukuba climb was thrilling

    This year we’ve resolved ourselves to climb Mt. Tsukuba. No big deal given the fact that we’ve lived near this mountain for past 10 yrs now. This mountain is one of the recommended mountain climbing choices in Japan. You can read more about this awesome mountain on Wikipedia.

    The map to climb Mt. Tsukuba is available as a PDF file here. The Miyuki-ga-hara trail is the most arduous while the Mukaeba trail is easier & much more fun together for first timers.

    Here are some photo memories from our climb up there – we chose Mukaeba trail to climb up, then used cable car down.

    Our second climb was more prepared (we were looking forward to the tough one this time). This time we chose to go up through the Miyuki-ga-hara trail, then chose Mukaeba trail back down. We didn’t manage to take many photos this time, but some of them are here if you’re interested.

    Definitely recommend this mountain should you choose to explore more from Japan!

  • How to increase HDD size for a VM

    Quick step by step guide on how to increase HDD size for a VM

    Steps required are,
    1. To increase the hard drive space, first increase the hard drive size in VMWare.
    2. Execute below commands to first recognize the partition
    # /sbin/fdisk /dev/sda
    p
    n, p, enter # select partition number
    enter
    p # to verify new partition
    w
    # reboot

    3. Add the new partition to total partition to increase the disk space
    # /sbin/pvcreate /dev/sda3 # Modify sda3 to the last sda
    # /sbin/vgdisplay | /usr/bin/head # Check the VG Name, and LV Name which we are going to extend
    # /sbin/vgextend vg_centos /dev/sda3 # Modify sda3 to the last sdan
    # /sbin/lvdisplay
    # /sbin/lvextend /dev/vg_centos/lv_root /dev/sda3 # Modify sda3 to the last sdan
    # /sbin/resize2fs /dev/vg_centos/lv_root
    # reboot

    4. You should now see increased disk space
    # df -h

    Steps have been adapted from
    http://www.bluhm-de.com/increase-centos-6.2-hard-drive-space-/-partition

  • Merry Christmas 2014

    Merry Christmas 2014

    Wishing everyone near-n-dear a very Merry Christmas 2014

  • What would future in the ‘cloud’ look like

    What would future in the ‘cloud’ look like

    I am always amazed at what computers have done so far. Many of us use, live and interact with a lot of devices, connected peripherals every second of our life. It has undoubtedly led towards a better, more social (perhaps), and a much engaging, but sophisticated ‘digital’ life.

    Whenever I look back over history on how this trend has progressed, the outcomes do seem to follow one particular pattern – progress, fueled by improving our ability to do some things we couldn’t accomplish so readily (and easily, economically) in the past. Things such as online shopping, commerce, ability to communicate, share important moments, bringing resources from one end of the globe to the other end in fraction of seconds. It is definitely a better place to be in today, than it was before. I believe most of us would now find it hard to imagine how could one live without an Internet, or even a mobile phone ever.

    One of the important achievements towards making this possible has been the big names such as Google, Amazon, IBM, Microsoft, HP etc pushing the limits of what software, hardware together can do to achieve a more connected, a more ‘alive’ digital world to come true. One of the most important topics of this decade (I would say) has been the advent of Cloud Computing.

    Cloud Computing, in a nutshell, is the ability to use computing resources – such as CPU, memory, storage – over the Internet. While the debate over whether Cloud Computing is just another market buzz is pointless in my opinion, it does has brought a few important questions to my mind as I think about what kind of future this progress can promise.

    Take a look at a mind-map I made recently when researching one of the most popular clouds offered – Amazon Web Services.

    Amazon Web Services 2014 Mind-Map

    The beauty of AWS (Amazon Web Services, for short) is how they’ve accomplished a feat with a clear mindset of making computing truly available as a utility. Of course, many in the corporate world would argue that such types of computing facilities already existed in virtualization world, and that is very true. Corporate worlds often had these needs to be able to spin out new machines, networks inexpensive yet keep economies of scale when you have to throw away any unused inventory that you no longer need. With a physical computer, it is hard to just get the right mix of specs to satisfy, but with a virtual one, things become slightly more manageable.

    One of the key strengths of the AWS, and perhaps the topic in question that I have in mind is, the ‘promise’ of utility computing, pay-as-you-go computing, ‘rent’ over ‘own’ type of computing.

    Historically, when computers were the size of a room, and mere mortals couldn’t afford (or even consider) to have one, IBM owned and dominated an industry of computing power available to corporations interested on a ‘rent’al basis. The premise was for IBM to offer computers to big companies, universities and let them use these under a pay-as-you-go sort of agreement. The idea was to be able to bill customers mainly on CPU time, but this soon expanded into an industry where it was a full-time job for one to understand, and perform accounting on CPU utilization, storage used, and even control the unwanted cost so that rented costs could be managed. I consider that this model had its run, and in some cases, is still prevalent in corporate worlds which use rental printers, infrastructure as a service, or some might say, even software as a service. This kind of model did have some benefits, mainly in terms of being able to not worry about maintenance of such complicated machinery, even not worry about servicing, maintenance since all of that could be bundled by IBM as a ‘service’.

    So, in a nutshell the benefits of using IBM’s model were

    1. You pay for only using what you used, nothing else. For a small cost, we take care of any hardware maintenance, patching, upgrades for you.
    2. No upfront costs, nor any costs down the line. We can offer, augment, decrease our rented resources to balance your needs
    3. Your environment, your assets. We give you the best computing resources to get things done cheaper, and faster.

    Of course, it may have been done differently for different sizes, different customers, different geographies, but I believe that was the promise expressed to alot of corporate consumers who started switching to personal computers, workstations before then. Of course, everybody wanted one computer of their own, and many still do.

    Now, when I look at AWS, I kind of feel many points are reminiscent of the time when powerful computing, networking, increased storage was not cheaply available for many of us to use. Networked computing, cost of managing servers, databases is hard, and not just that, it is ‘expensive’ too once you own it. Given the vast power offered by Amazon to anyone at their fingertips, many young entrepreneurs can surely do wonders. But if you consider for a moment about what goes with becoming together as a part of this ecosystem, you do have to question yourself – if you do wish to ‘pull-out’ at some point, would it be possible for you to do? The simple answer could be yes, but as with many things in life, an investment into something, even if it is a cloud, is not so easy (or practical) to undo.

    I predict that as with many trends within digital world, cloud computing will bring about a division within the digital world with big corporations driving market share towards cloud, yet still keeping their heads occupied towards new management issues resulting of out ‘wasted’ cloud usage. Similar to electricity savings during summers, there will be CoolBiz days where people are encouraged to optimize their ‘spend’ on utility computing. New jobs requiring cloud administration, monitoring, usage accounting on cloud will spring up. We’ll decide how advanced is a country by looking at its consumption charts on daily computing used compared to rest of the world. Most probably, utility computing might even become a government-owned service offered to anyone who would contract, and pay for its utilization.

    It’s not too often I compare the scales of performance that everyone sees of Amazon, Facebook, Google compared with their own, and be somewhat mentally assured about the fact that if you’re on the cloud, you can have all of that anytime. What many of us do fail to acknowledge is that when you’re the size of Google, Amazon or Facebook, you’ll have enough incentive to use a cloud, or even with the right team, have a mix of your own together with public clouds together. Whether we do have the incentive today or even now is definitely not an easy question. But a ‘lock-in’ into any kind of technology, be in PCs, cloud is always going to have implications on future for rest of us.

    As the digital world continues to become an increasingly complex world of its own, giants like Google, Amazon who are best are what they do will continue to fuel it. Whether one sees this as an opportunity, a trap, an evolution in networked digital computing is best left to people. But from an altogether different perspective, I believe one must see beyond the promise painted today into what it can become tomorrow.

    My advice – definitely exploit the cloud computing but in moderation. You’ll only want to live in a shared/rented apartment, no matter how good, for so long until you can finally own one of yours.

    I think most of us eventually do own one.

  • Good artists copy; great artists steal

    Good artists copy; great artists steal

    Many of us associate this rather unethical quote with Steve Jobs of Apple. At least that’s where I heard it first. The second time around I came across this quote was in Cringley’s excellent book Accidental Empires.  The opinions are no doubt sound, and does make one ponder about the real experiences these famous entrepreneurs went through before getting at where they are.

    I must admit, I liked the final passage about computing as a transitional technology. The actual paragraph as quoted from Cringley’s book goes something like this-

    We overestimate change in the short-term by supposing that dominant software architectures are going to change practically overnight, without an accompanying change in the installed hardware base. But we also underestimate change by not anticipating new uses for computers that will probably drive us overnight into a new type of hardware. It’s the texture of the change that we can’t anticipate. So when we finally get a PC in every home, it’s more likely to be as a cellular phone with sophisticated computing ability thrown in almost as an after thought, or it will be an ancillary function to a 64-bit Nintendo machine, because people need to communicate and be entertained, but they don’t really need to compute.

    Computing is a transitional technology. We don’t compute to compute, we compute to design airplane wings, simulate oil fields, and calculate our taxes. We compute to plan businesses and then to understand why they failed. All these things, while parading as computing tasks, are really experiences. We can have enough power, but we can never have enough experience, which is why computing is beginning a transition from being a method of data processing to being a method of communication.

    People care about people. We watch version after version of the same seven stories on television simply for that reason. More than 80 percent of our brains are devoted to processing visual in formation, because that’s how we most directly perceive the world around us. In time, all this will be mirrored in new computing technologies. We’re heading on a journey that will result, by the middle of the next decade, in there being no more phones or televisions or computers. Instead, there will be billions of devices that perform all three functions, and by doing so,will tie us all together and into the whole body of human knowledge.

    I must admit the opinions are very plausible. It is more “correct” to consider technology as a transition than destination. Indeed … ! Success does look planned in retrospect.

    Yet another context set off echoing my own thoughts in the same book, which goes like this-

    It was in the clay room, a closet filled with plastic bags of gray muck at the back of Mr. Ziska’s art room, where I made my move. For the first time ever, I found myself standing alone with Nancy Wilkins, the love of my life, the girl of my dreams. She was a vision in her green and black plaid skirt and white blouse, with little flecks of clay dusted across her glasses. Her blonde hair was in a ponytail, her teeth were in braces.

    ‘Run away with me, Nancy,’ I said, wrapping my arms around her from behind. Forget for a moment, as I obviously did, that we were both 13 years old, trapped in the eighth grade, and had nowhere to run away to. ‘Why would I want to run away?’ Nancy responded, gently twisting free. ‘Let’s stay here and have fun with everyone else.’ It wasn’t a rejection, really. There had been no screams, no slaps, no frenzied pounding on the door by Earl Ziska, eager to throw his 120 pounds of fighting fury against me for making a pass at one of his art students. And she’d used the word let’s, so maybe I had a chance. Still,Nancy’s was a call to mediocrity, to being just like all the other kids.

    Running away still sounded better to me.

    What I really had in mind was not running away but running toward something, toward a future where I was older (16 would do it, I reckoned) and taller and had lots of money and could live out my fantasies with impunity, Nancy Wilkins at my side. But I couldn’t say that. It wouldn’t have been cool to say, ‘Come with me to a place where I am taller.’

    We never ran anywhere together, Nancy and I. It was clear from that moment in the clay room that she was content to live her life in formation with everyone else’s and to limit her goals to within one standard deviation on the upside of average. Like nearly everyone else in school and in the world, she wanted more than anything else to be just like her best friends. Only prettier, of course.

    Fitting in is the root of culture. Staying here and having fun with everyone else is what allows societies to function, but it’s not a source of progress. Progress comes from discord—from doing new things in new ways, from running away to something new, even when it means giving up that chance to have fun with the old gang.

    To engineers—really good ones, interested in making progress—the best of all possible worlds would be one in which technologies competed continuously and only the best technologies survived. Whether the good stuff came from an established company, a start-up, or even from Earl Ziska wouldn’t matter. But it usually does matter because the real world, the one we live in, is a world of dollars, not sense. It’s a world where commercial interests are entrenched and consumers typically pay closer attention to what everyone else is buying than to whether what they are buying is any good. In this real world, then, the most successful products become standards against which all other products are measured, not for their performance or cleverness but for the extent to which they are like that standard.

    It is like looking straight in the mirror talking to myself. How many times do I feel the same way? Mediocrity is considered normal, and anything that challenges that, even sarcastic humor, is seen as an insult or even a challenge to intellect blinded by a zeal to fit “in” no matter what it means for oneself.

    A good memo of the thoughts that concern me as an entrepreneur set out to change the very same world I’ve lived in.

  • Which works best – centralization, or decentralization

    Only the Paranoid Survive by ex-Intel CEO Andy Grove was a wonderful introduction to me about the importance of keeping your organization’s opportunities in check with ever-changing market needs. It did give an importance to diversifying centralization towards specialized decentralization – vertical industries against horizontal industries.

    Somehow at that time, which was almost 6 yrs ago, it seemed correct and proven with horizontal names such as Microsoft, Dell, Intel, other OEMs. In fact, I learnt most of my PC maintenance knowledge getting to put these together, and it still works great. The brilliance of plug-n-play type of PC industry really meant a lot of collective good coming together as one for benefit of the customers. And it still rules.

    On the other hand, Apple. For a long time, I kept resisting Apple. I was happy with my world as being able to choose from different options (was I really?) over being bundled together into one by Apple. Somewhere, I switched to iPhone, and my views expanded. I unconsciously started to measure ‘good’ in it’s entirety.  Not just the idea of ‘monopoly’, but the actual real-world example of that, the influence it showed to an industry I knew was the best, and perfect was dramatically changed.

    Now, in my own words, Apple is different to what Microsoft is. By that I literally mean their idea of ‘selling’ themselves. Microsoft enabled PCs, enabled OEMs, enabled plug-n-play but left out one very vital thing – their administrative burden was given to the people. Suddenly we had device drivers, certified programs, expertise sprung around specialization. I had to ask myself – were we really decentralized? You basically have to know your PC before getting to use it. I was a technician, and a user both at the same time. Essentially, I had to make things work by taking what is offered by horizontal industries, and hope to not touch it again (windows upgrade?).

    With Apple, I was no longer a technician. Hell, I tried to find Task Manager the first time I handled my iPhone. Slowly with each day I start to ‘use’ my iPhone without bothering that I’d have to take it apart. I was a pure consumer. And that was not just hardware, it was software too. In the back of my mind, I am happy that things funnel through Apple. I am tempted to go beyond, and take things in my hand, but I resist. I remind myself of where I should focus, and what is important.

    So in a sense, I prefer the horizontals, but funneled through verticals. In many ways I find that Apple shows responsibility for whatever it offers. It will use horizontals – vendors, makers, app developers – behind its strong fort, but what comes out is from Apple as a single contact with me. It makes me wonder if horizontals did really catch the drift correctly? In many ways, many still want to reach out directly to the customer in trying to push customers into the driver’s seat. Customer’s however seem to be comfortable either way … (alas)

    I favor responsibility going beyond just my product. I believe Apple (through Steve Jobs) realized that goal, leaving it to continue as a better way of doing good to the world at large.

    In some sense I trust the paranoia hinted by Andy Grove, but I believe too in enduring excellence. I endeavor to achieve that as a responsible entrepreneur someday.

  • cld2 – Google’s Compact Language Detector 2 – standalone command line on Cent OS

    It appears that cld2 has no mention of how one would go about using it (or at-least that is the way it looks to me). The language detection ability is one of the better ones, and I decided to make use of it.

    I came across a blog mentioning how to install cld2 on ubuntu but it just fell short of using it directly through a command line. It mentions how to build a Python binding.

    Luckily, I also came across another blog where a Slackware script mentions building a command line tool which is perfectly what I was looking for, except that I had CentOS, not Slackware.

    So with a little bit of digging around the various compile scripts on cld2’s SVN trunk, I got a faint sense of combining the ideas from these two blogs, and give it a try. I succeeded! Here’s what I did

    1. Get g++, it is required to build cld2 on your CentOS machine
      $ /usr/bin/sudo /usr/bin/yum install gcc-c++
      ...
      $ which g++
      /usr/bin/g++
      
    2. Get the cld2 source through SVN on your local CentOS machine. In my case I used /tmp folder
      $ pwd
      /tmp
      $ svn checkout http://cld2.googlecode.com/svn/trunk/ cld2
    3. Next, make a copy of one of the already existing compile scripts to make a few changes, specifically compile_libs.sh. The step is mentioned already in how to install cld2 on ubuntu. I use 32-bit, hence I use the same step remove the -m64 flag.
      $ pwd
      /tmp/cld2/internal
      $ cat compile_libs.sh | sed 's/\ \-m64\ //g' 1> compile_libs_32bit.sh
      
    4. To make a standalone cld2 executable, again I followed the steps from Slackware script example. I made following changes to my copied compile script. Here’s a diff of what changes I made from compile_libs.sh to my custom compile_libs_32bit.sh script
      https://gist.github.com/visitsb/8affec514ef5829c6bd0/revisions
    5. That’s it! Now compile_libs_32bit.sh is ready to build a standalone cld2 executable on your machine. It is just a matter of executing your custom compile_libs_32bit.sh script now
      $ chmod u+x compile_libs_32bit.sh
      $ ./compile_libs_32bit.sh
      
    6. It takes a few mins to build, and voila, you have a standalone cld2 executable built, and installed on your machine.
      $ which cld2
      /usr/local/bin/cld2
      $ echo "Hello World こんにちは γει? σου" | cld2
      ExtLanguage Japanese(35% 3904p), GREEK(33% 1024p), ENGLISH(27% 1194p), 45/43 bytes of non-tag letters, Summary: Japanese*
        SummaryLanguage Japanese(un-reliable) at 8391021 of 43 562us (0 MB/sec), (null)
      
    7. For the record, here is what get’s installed
      $ which cld2
      /usr/local/bin/cld2
      $ ls -l /usr/include/cld2/*
      /usr/include/cld2/internal:
      total 52
      -rw-r--r--. 1 root root 28159 Jun 20 17:49 generated_language.h
      -rw-r--r--. 1 root root  5839 Jun 20 17:49 generated_ulscript.h
      -rw-r--r--. 1 root root   945 Jun 20 17:49 integral_types.h
      -rw-r--r--. 1 root root  8326 Jun 20 17:49 lang_script.h
      
      /usr/include/cld2/public:
      total 24
      -rw-r--r--. 1 root root 14850 Jun 20 17:49 compact_lang_det.h
      -rw-r--r--. 1 root root  7056 Jun 20 17:49 encodings.h
      $ 
      $ ls -l /usr/lib/libcld2*
      -rwxr-xr-x. 1 root root 6457627 Jun 20 17:49 /usr/lib/libcld2_full.so
      -rwxr-xr-x. 1 root root 1742462 Jun 20 17:49 /usr/lib/libcld2.so
      $ 
      

    Hope this helps someone, and kudos to cld2 for being awesome!