npm and trust

I don’t keep up much with security bulletins, there are too many of them to be of more than statistical interest to me!

But I did hear about this one with an quiet little npm package that spread to a few million places in an attempt to do some cryptocurrency hacking.

Read the news version on ars technica or the github thread (complete with many a flame).

I don’t think that there’s an easy solution to it; a few people noted that probably handing over the repo should have been a major change in the npm version, others noted that the original dev should have forced the new maintainer to fork. But that’s the way hacks go right? Easy to say in retrospect, but maybe that bad actor approached dozens of package owners and only one had to slip up just once.

Of course the most brilliant of our grandparents of compsci already knew this: Reflections on trusting trust (1984) describes an attack on a compiler that produces programs with a backdoor, and also compromises the compiler when it compiles another compiler; and the debugger and the linker so it hides its own existence. As any good student of crypto knows, you can’t trust any program from source code unless you trust the toolchain it was built with. (And since meltdown etc. does that also mean the hardware too?)

And we also know “all bugs are shallow“.. except of course that is only strictly true in places where there there is a greater number of people trying to keep the bad actors out than there are bad actors… (or is it greater number of more paranoid people..?) Given that governments employ some very good hackers, can we honestly expect lone Open Source devs to stand against state-level adversaries… surely a little light burglary would be enough to compromise many a npm package…). And what about heartbleed, surely the most watched piece of Open Source code?

So, where is your confidence level on your npm and nuget packages? What about even client-side javascript dependencies (since there must be exploitable browser vulnerabilities)?

Or do you get confidence by blocking every port, keeping servers off the internet and watching those logs for suspicious stuff?

Advertisements

The Agile Retrospective

The great thing about any type of iterative project is that the repeating cycles give you a chance to make mistakes and learn from them while that learning is most useful. When you do one big project and the failure comes at the end, you have no chance to set it right. The next project might not be similar enough for the lessons to seem relevant.

With an iterating project you can take a small risk now and pay it back next time if it doesn’t work out. You can steer onto the right track while there is still time and the cycle also gives a natural break point to reflect.

In the short period of the agile cycle, you can see what you did for yourself without a post-project review, the backlog progress is so tangible and SMART. But the agile retrospective also demands something that no post-project review ever did: it needs candour. Jack Welch hit it on the head when he says everyone is honest, but no one is candid.

You need to be able to your team in the face and say: “I could have done that better”, not defend what you did. The fact you have a chance to change next week gives a freedom to be honest, it lowers the barrier that we all have to admitting our mistakes. But you will also be getting candid feedback, from people who know you and are very familiar with the work. The key thing is that you receive that feedback from a position of self-confidence; this isn’t a year-end review, you aren’t going to be fired. There is always next month to change, or if things really aren’t working out after a few months, maybe you slide over to doing something slightly different.

But you also have to give people feedback correctly. You aren’t RMS telling someone off for having a baby; if the culture that your senior staff have cultivated in your team is one of brutal smackdowns on junior staff then good luck to you. It’s possible you are all very happy and I hope that works for you but I’m out.

You need to be direct with people but you don’t need to be blunt or aggressive. Cultures vary though, and in my experience my Englishness has not prepared me for the level of directness that is OK in many other European countries*.

Ask don’t tell:

  • instead of “I think you did that wrong” try asking: “Can you tell us how that went? Why did it turn out like that? what was difficult about it?”
  • Trust your colleagues to have done the best they could, even if (say) on this occasion they put delivery over quality and that wasn’t the right choice because it broke the build for a week

You should critique the work, but not the person

  • Think carefully before you say “you should have done that better..”
  • Maybe: “I think that code could have been better”
  • Ideally be specific as well “I think that class was too big because it includes this and that”

Set your sights very high, but don’t expect to be there next week.

  • Agile allows for limitless improvement.
  • Don’t accept things as they are, don’t say “yeah the merge took 2 days, but it was a hard merge”; say “that merge was hard, it took 2 days, it should have taken 20 minutes. How can we beat that?”

One of the most important things for learning is stability. Being able to isolate important variables and ignore unimportant variables is basically the cornerstone of the scientific method; from learning to do a handstand to getting a good result in a physics experiment. If you go through a stable, repeatable process that gives consistent feedback close the point of the event, you have the best chance of that feedback taking effect**. One of the most stressful things to experience in work is so much unpredictability that you don’t know whether your efforts are making a difference to outcomes.

*And remember, I was a physicist. When I did physics it was acceptable to end a conversation with a frown and then just turn and walk away.

** see also: intermittent variable reward; you might think that this is a great way to ingrain habits, but I think it’s a great way to compound neuroses and a great way to make team members compete with each other. IVR is why twitter bullying and gambling are so effective at ruining lives. Kathy Sierra here on that topic: http://headrush.typepad.com/creating_passionate_users/2007/03/is_twitter_too_.html whose site was famously closed by online harassment.

Charity

I’ve never been much of a charity person, I’ve always found it hard to donate to a cause but recently I found a cause that worked. For me, it wasn’t an international crisis, it was a local problem and it wasn’t a donation of money.

My children’s pre-school had a website, the person that made the website was looking to move on from the responsibility 5 years later. The pre-school wanted to make some changes to the info on the pages and didn’t know how. When I took a look at it, it was obvious why. The site was hand-coded HTML, and every time a phone number or anything needed changing they had to call the IT person. And that person allegedly ran a website company. Shocker.

Well that just was not going to continue. Not on my watch.

So that was where I made my charity donation. It was all about using what I knew effectively. For me, the effort to set up hosting and installing the mighty WordPress was all a tiny amount of effort. It was effortless to immediately do a risk analysis of security and put in change control practices, to start doing backups and to choose a software package that permanently solved their problem, and solved it so well that it transformed what they could do.

However, I had to exert myself to not do too much. Interesting how keenly you feel the support work when you have to do it at 1am and that isn’t even your real job. But. Also, how sweet the saccharine hit of solving a problem that is totally out of reach, and solving it for an organisation that is caring for your child. They have no time to sit down and solve an IT problem and every pound they spend on a support contract with an IT firm is a pound they don’t have to spend on another teaching assistant.

Of course, it spiralled a bit. Soon they had some email problems, and laptop problems and needed to change the domain name and they kept me on the hook. Occasionally it felt too much, but that’s their job squeezing the lemon. And they have come to rely on me for a few things. But then, out of the blue, someone on the governing committee just said “I’m going into bat for Team Sam, I’m getting someone else to do it.” And they did.

I got to experience the final stage of the charity project: the hand off to the next volunteer. I even wrote the documentation! It wasn’t The Cathedral and the Bazaar (point 5), but I have to say I felt a tiny bit of pride that I’d applied global tech firm, auditable standards to this tiny, tiny IT problem and I could just hand it on knowing that things were in great shape.

For me, donating to people who needed help was great, but I also kept myself sharp. I didn’t use my normal technologies, it was a different problem and had to be running without support for years as “good enough” rather than our own multi-million pound efforts to get to 1000% note-perfect auditable polish. That was a great experience. I may have emitted the occasional curse while on the journey of course…

(this is an older post recycled from 2010)

Changing fonts in R

I’m trying to make R charts that meet my company’s brand guidelines.

I’m trying to get different fonts into R,  you need to load them into R separately as it can’t just see the windows fonts. But I suffered so you don’t have to.

I had a couple of options for the package that I used, and I used this trick to compare the packages. There’s a useful package called cranlogs that allows you to see how much they get downloaded, which is an indicator of their support level and freshness! Here i compare the showtext and extra font packages.

install.packages("devtools")
library(devtools)
devtools::install_github("metacran/cranlogs")
cranlogs::cran_downloads(when = "last-month", 
                         packages = c("showtext", "extrafont"))  %>%
  as_tibble() %>% 
  ggplot(aes(x = date, y = count, color = package)) + 
  geom_line()

I got more help on extrafont here.

Next: onto the colour wheel and loading the brand colours into the colour palette!

results:

image2018-7-24_18-1-39.pngimage2018-7-24_18-2-0.png

 

install.packages("extrafont")

library(tidyverse)
library(extrafont)

# have to run this once only per system
# takes several minutes to import them all
# after that it loads all the fonts when you 
# load the package
# if you uninstall the package, you have to do 
# it again...
if(! "Gotham Office" %in% fonts() ){
  print("You haven't imported the fonts into R")
  print("This will take a few minutes")
  print("You only need run this step once!")
  font_import()
  loadfonts(device = "win")
}

cheeseOrFont <- function (font_family){
  ggplot(mtcars, aes(x=wt, y=mpg)) + 
    geom_point() +
    ggtitle("Fuel Efficiency of 32 Cars") +
    xlab("Weight (x1000 lb)") + 
    ylab("Miles per Gallon") +
    ggtitle("NOW YOU CAN SEE that the font is changed") +
    theme(text=element_text(size=16, family=font_family)) 
}

cheeseOrFont("Gotham Office")
cheeseOrFont("Comic Sans MS")

How fast can you form a team?

When we look for excellence, looking at extreme examples can help to change expectations. When building a team, we expect that that it takes time to move through stages until the team becomes “performing”. We expect that to take time; many weeks to months.

What about forming a team in an hour? Can that be done? A team that collaborates, with individual contributors making their voice heard, each person bringing their own skills, each person working outside of their comfort zone? Is that possible in an hour?

My experience of jury service showed me that it is possible, but: can we replicate that in our own teams? What are the causes? What are the limitations?

Juries in the UK are drawn from the adult population; anyone on the voting register can be selected. That means that the jury is diverse. In our case we had a couple of people in their 20s, and up to people in their late 60s. We had, men and women; professionals and non-professionals. Retired accountants who were school governors, carpenters who were part time firefighters, grandmothers who stack shelves in supermarkets, teachers, project managers, sick and healthy, old and young. So you could argue we had no useful skills; but really on a jury the only expertise is life. The more diverse your life experience the better you are at seeing in the motivations and excuses that are the bread and butter of a court case. You might argue that experts in justice would be better, but the fresh approach that a new jury brings means that trials are heard fairly; if you spent all your time in court cases where half the people were lying, you quickly slip into cynicism. Also, evidence in court cases is rarely complete; you have points of light from physical evidence but even that can be interpreted. The more diverse a team is, the more angles they can examine from. Diversity is strength.

Key point: Juries are fresh and enthusiastic. Juries are made of diverse individuals.

 

One thing that helps juries bond is the seriousness of the situation. From the moment you step into the court you are under no illusion that these people are not messing around: the silence, formal etiquette and formal dress code all reinforce that. The judge leads the court, both the jury and barristers. The instructions given to jurors are unflinching in their seriousness: that we are expected and required by law to make a good job of this. We take no breaks except when given leave by the judge. There are legal penalties for failing to attend. However, the judge also gives relief to juries; they need not know the law, they need not concern themselves with anything but the evidence; and in the evidence they have free rein to interpret it anyway they like. Even if the barristers or the judge give advice, in matters of evidence, the jury may ignore it. You’d think that there would be little room for interpretation of evidence and the law, but no case gets to court unless it is complicated and subtle. If there was clear physical evidence then the case would not be heard at trial. For a jury, there are no facts only interpretations.

However, even though juries need not know the law, the evidence is a heavy burden. Outside we might joke about how “they are all guilty”, but in the court you never joke like that. There is no doubt there is a moral duty, to the victim and the accused. We are asked to aspire to the highest standards; way above what we follow in our day to day lives.

Key point: the judge gives clear and unambiguous executive leadership; setting standards and burdens way above our expectations, then asking that we rise to them.

 

But how do juries form into a team? Especially one that has no time to form, and little common ground outside the court setting. We aren’t allowed to go out together and discuss the case (in fact we aren’t allowed to discuss the case at all until evidence is finished). Then we aren’t allowed to do anything else other than discuss the case. We can make only small talk until the discussions start, then we have to go into a room and only come out to take bathroom breaks and to go home if there is no decision. We all had to go for cigarettes together, even those who didn’t smoke. There can be no discussion that is not heard by all, and no discussion that can be heard by anyone who isn’t on the jury.

In my experience – and I can’t say too much about what we discussed –  the high standards we were set encouraged us to  all respect each other, and allow everyone time to speak. We quickly created a minimal process for round table discussion and everyone had something to say. In juries, all are created equal; education, race, age, creed all are relevant but none grants superior point of view. All are equal, everyone has the right to speak, none has the right to compel anyone. As discussion progressed, some people brought their talents to bear; technical skill helping others examine CCTV evidence, patience in balancing argument, being willing to lead a discussion.. Or stop one. Creating mechanisms to progress discussions and drive consensus.

Key point: clear rules of equality are established, then let the team run itself.

 

The output of a jury is clear, and is commanded clearly by the judge: first prize is a unanimous decision. If agreement can’t be reached, then a majority decision of 10:2 is sufficient. We all know what we are trying to achieve and all of us an equal stake in that. For my own part, we worked together bringing our own viewpoints but my verdict was my own. It was painful, but reaching my own decision was accompanied with a sense of peace and achievement that contrasted sharply with the uncertainty and weight of the process.

Yes, of course there was disagreement. Probably the worst of which was – I’m not proud to say – caused by me. What’s the only thing you can do wrong on a team trying to achieve a consensus? You can try and block or influence someone else with the force of your personality rather than the quality of your argument. I overstepped the line and was put in my place, immediately. I wonder if we would have been able to manage individual underperformers or lack of engagement; the matter didn’t arise the situation and the case so serious.

When we reached our decision, many people tried to express their feelings on being on the jury; it wasn’t one of enjoyment as such. Put a powerful sense of fulfilment; high standards had been reached and we’d really done something and done it together. Mostly people spoke about how proud they were.

Key point: when expectations are above and beyond, people don’t want to be the one that lets team down.

 

So why don’t all teams form this way? For starters, this environment of genuine importance can’t be faked. People take it seriously because it’s serious. Young lives are being ruined on both sides of crime and that gives depth to the experience. Other careers may experience similar moments of crisis, but crisis can’t be faked. The intensity can’t be maintained in a work setting; not everyone is dedicated like that to their career and the imbalance would quickly create tension that would not be resolved. The unity of purpose of a short sprint meant that the diverse team could hold together for that period but it would not be sustainable.

The following week, I was on a second trial. It wasn’t jerry springer but compared to the violence and chaos of the first case it was positively recreational. That allowed us to step down gently, and return that amped up feeling of importance to normal.

Key point: put teams under pressure to achieve exceptional results; but don’t try and maintain those sprints. It can’t be done. Allow teams time to unwind and process the experience they’ve been through.

RStudio nearly has python, d3 and others

I’ve been playing around this week with python and d3 in vNext of Rstudio. The next version will be 1.2 and is intended to be out later this year.

The upgrade didn’t go that well.

Firstly I had to patch the drivers for my laptop’s graphics card or it didn’t work (actually, the screen was white when i connected my multiple monitors…)

Then I had a whole other set of problems with my python installation, which basically came down to “put it on the path, at the top”.

Then we had some fighting between Miktex (for PDF rendering), Anaconda and something else. Which means I have to go back to Rstudio 1.1 for my real work.

However, it was cool to see R and python together in one document, and able to access the same data. And d3.. well I think that’s going to be useful for the more interactive style graphics.

Gitting better at git

I use git for my work with RStudio but only in a very crude “click this and then click that” way. You know the ritual: stage-commit-pull-push and pray that nothing goes awry.

Of course, anyone wise knows that you have to know git on the command line. Not to be a guru, but to be effective.

I’ve been working with this book today: https://leanpub.com/learngitthehardway and it’s excellent. I bought my own copy.

After about an hour with the book, I checked in from the command line from my embedded powershell terminal inside RStudio, and didn’t I feel like a grown up?

What’s good about it?

  • lean and brief; well written
  • gets you working within 5 minutes
  • dispels fear by repeated practice

Caveat: It’s deeper than my knowledge, so I can’t tell if it contains the deepest advice. Also – for a windows dev – I’ve been on the command line a fair bit.

I’m recommending this book to everyone who goes near git.