UW Spring Security

03 Mar 2016

Here are slides from a talk I am giving on Friday, March 4 2016:

This talk is focused on a project my colleagues and I have been developing over the past year called uw-spring-security. uw-spring-security is a multi-module Java project that University of Wisconsin applications can use to secure web applications in a way that is compatible not only with our single-sign-on technology, Shibboleth, but how we deploy it.

I have used Spring Security for many years, and in my opinion, it is exceptional. It has comprehensive support for all things authentication and authorization. The only drawback: it is terribly difficult to get started with. The reference manual is 36 chapters. Just the getting started chapter (chapter 3) is a long read, and the topics covered aren’t relevant to what University of Wisconsin Java developers need to know.

Pairing Spring Security with the way we deploy Shibboleth is not intuitive. uw-spring-security addresses that by replacing the boilerplate configuration classes described in the Spring Security reference manual with:

@Configuration
@Import(UWSpringSecurityConfiguration.class)
public class MyWebSecurityConfiguration {
  @Bean
  public HttpSecurityAmender httpSecurityAmender() throws Exception {
    return new HttpSecurityAmender() {
      @Override
      public void amend(HttpSecurity http) throws Exception {
        http.authorizeRequests()
                .antMatchers("/protected").authenticated();
      }
    };
  }
}

Starting a web application with that @Configuration properly registered will result in seeing a login form when hitting the /protected URL.

If the entire application requires uniform authentication, there’s an even simpler example:

@Configuration
@Import(EverythingRequiresAuthenticationConfiguration.class)
public class MyWebSecurityConfiguration {
}

Determining whether Shibboleth (pre-authentication) is enabled or a local login form is used is accomplished by activating one (or both) of the 2 Spring @Profiles the library supports: preauth or local-users, respectively. Both profiles present the exact same data model for a User:

{
  pvi: "UW000A000",
  username: "admin",
  password: null,
  fullName: "Amy Administrator",
  emailAddress: "amy.administrator@demo.wisc.edu",
  uddsMembership: [
    "A535900"
  ],
  authorities: [ ],
  accountNonExpired: true,
  accountNonLocked: true,
  credentialsNonExpired: true,
  enabled: true,
  eppn: null,
  emailAddressHash: "b09ed4fa2272feede8b472d1184829dd",
  source: "local-users",
  customLogoutUrl: null,
  isisEmplid: null,
  firstName: null,
  lastName: null
}

This library has been immensely useful within my group, and now it is time to share with a wider audience.

This library paired with uw-frame and rest-proxy gives us all we need to create business applications in My UW Madison. These 3 components provide all of the application middleware, and allow us to write user interfaces simply with HTML, JavaScript and AngularJS.

Visualizing Development, Take 2

02 Dec 2015

The following are the slides from a talk I am giving on Thursday, December 3, 2015:

Visualizing Development

The motivation for the talk was a follow up to the Visualizing Collaborative Software Development post and related video I published previously.

Looking back at that original post, I think the video does a good job of showing collaboration, but I think it falls short in demonstrating the actual size of the assets we manage. Each of the dots is a file, but no dot is larger than any other. I don’t think it’s fair to show a 20 line file and a 10,000 line file as having the same size.

This new approach tries to relate the size of our projects’ codebases to the novel Anna Karenina by Leo Tolstoy. There are a lot of similarities to made between writing code and authoring a novel. I think we all have a reference point of the mental effort and time required to read a novel. It’s not a stretch to relate those same mental activities to reading code - either someone else’s or even our own from a few months back. With contemporary sofware engineering tools and techniques, the scale of what an engineer needs to understand escalates exponentially.

This talk’s first audience includes the leadership of my current organization. My hope for presenting the talk is to recommend a few ideas regarding how we maintain our software projects:

Be more transparent. Code really should be visible at the organization level. We gain nothing by hiding from ourselves, we only introduce a high potential cost that is realized with staff change - not just turnover, but re-assignment too.
Write more that isn’t “code.” Executable/deployed code by itself is only a small part of IT solution delivery. Without context, that code can be unintelligible, as the last slide in the bonus code samples. Documentation and unit/integration/e2e tests are really the things that capture the business need of why we worked on a project in the first place, and can even persist through multiple different implementations. Version control history is critical too, because it lets us capture what we’ve tried before, particularly when we write good commit messages.
Curate. Dedicate more time to keeping things buildable, deployable, repeatable. If we’ve done 1 and 2, it will be easier for groups with cross cutting focus (deployment, security, testing) to interact with our IT solutions. That can even facilitate those groups contributing fixes or patches without waiting for a developer to be available to unlock access to the code.

Additional Resources

Google presentation on the scale of their codebase. Watch this video, it’s worth the 30 minutes. Teaser: Google’s internal source is managed in one large repository, shared with the company, with over 1 billion files and over 2 billion lines of code. That is 46,296 Anna Kareninas.
Anna Karenina metrics courtesy of Visualization of the complete text of Anna Karenina.
Bonus code samples courtesy of Obfuscating ‘Hello World’ in Python.
Lines of code metrics courtesy of cloc.

Visualizing Collaborative Software Development

04 Nov 2015

I’ve been told that from a non-technical observer, it’s difficult to understand what software engineering is.

As the technical lead for a team of engineers, I’m regularly tasked with explaining or justifying the effort behind custom software development projects, from “small” to “large.”

There are a number of different methodologies for describing the relative difference between a small and a large effort in technical terms, but those methods are lacking in clarity for a non-technical observer.

Some projects have an incredibly large but simple feature set that can be provided with a relatively small code base. Other projects have a seemingly tiny feature set, but have enormous complexity and large amounts of code. To a non-technical observer, it can be difficult to comprehend how a seemingly “small” web site can require so much development effort to deliver.

Visualizing Relative Scale

I’ve known about the existence of a tool called Gource for some time, and experimented with it a few times. I had an idea recently to use Gource to demonstrate:

relative size and complexity of the projects in our portfolio, and
how our engineers work together on those projects

My hypothesis: a visual representation of the activity of the team may provide a non-technical observer a better understanding of the relative difference of effort between software projects.

Part of my team’s current portfolio is made up of about two dozen different independent software projects, primarily web applications. Those applications are composed of different technologies and are different stages of the lifecycle, from green field through end of life.

Here is the output from Gource on the combined activity for those two dozen projects between June 2014 and late October 2015:

Description:

Internal Applications team members are shown using their avatars. Non-team members are represented with the little chess piece icon.
Department and division web sites my team regularly constructs are not included in this demonstration.
Each path radiating from the center point represents a different project in our portfolio.
Each leaf dot is a file within the project, only file extensions are shown to show the diversity of languages and content we must produce.
Each laser beam from an avatar to one or more dots represents an individuals contribution; each bug fix or feature we develop results in adding or removing 1 to thousands of files.
On Monday, August 17, 2015 we imported a software project that has been maintained without verson control for more than a decade. The primary developer for that project transferred from another part of the organization to our team a few weeks prior.

My Takeaways

Collaborative development provides enormous value to a project, particularly as the project increases in size. Any engineer from our team - and even engineers outside of our team - should be able to contribute to any project in our portfolio. I will write a future article on the technology strategy and tools we use to accomplish that. You can see each multiple different members contributing to each of the different projects and pathways.
Our biggest challenge: our engineers are outnumbered by projects. This video could look completely different:
- If each engineer worked solo on projects, their avatar would stick on top of one portion of the tree and never move. We would never have the benefit of different perspectives contributing across the portfolio, and we’d have difficulty in cross training or filling gaps if an employee leaves.
- If we could focus the entirety of the team on single project for longer contiguous periods of time, you would see less dancing from project to project. We are losing efficiency with all of this context switching; I will write a future article on our team’s ongoing method to address this.
August 17, 2015 is a watershed moment for our team. As demonstrated by the video, the sheer size of that project is greater than all of our existing projects (in version control) combined. Simply switching from a single-developer to a collaborative model is no small feat for this project, it will take us significant time and effort to transition that project.

I hope to re-visit this technique on a yearly basis.

The source material used to create this video can be found at https://github.com/nblair/internal-apps-visualization.

Configuring GPG/PGP for Maven Releases to Sonatype on Mac OS X

29 Oct 2015

Each time I go a long stretch without publishing a Maven artifact via Sonatype, I find it’s easy to trip up on the GPG configuration, particularly on Mac OS X. I’m recording this here so hopefully the next time it’s a little easier.

Requirements

Assume Homebrew is installed.

brew install gpg2
brew install gpg-agent

After gpg-agent is installed, you’ll want to tweak your shell via it’s rc file (I use oh-my-zsh). Add the following at the end:

eval $(gpg-agent --daemon --no-grab --write-env-file $HOME/.gpg-agent-info)
export GPG_TTY=$(tty)
export GPG_AGENT_INFO

Create and publish your Public Key

You don’t have to do this every time. Run gpg2 --list-keys to see what you have. Publish a key only if you don’t have any keys or if they’ve expired.

Sonatype has good documentation for this already.

TL;DR version:

gpg2 --gen-key. Yes, you should encrypt your key with a passphrase, remember what you used, you’ll need it later.
gpg2 --keyserver hkp://pool.sks-keyservers.net --send-keys keyid

Configure Maven

We need to pass properties into the maven-gpg-plugin. Putting them in the project’s pom is a terrible idea, and passing them on the command line is awkward.

A better place is in our ~/.m2/settings.xml; add the following to your <profiles> block in that file (and here’s the reference if you don’t already have a custom Maven settings file):

<profile>
  <id>gpg</id>
  <activation>
    <activeByDefault>true</activeByDefault>
  </activation>
  <properties>
    <gpg.useagent>true</gpg.useagent>
    <!-- gpg-plugin defaults to trying 'gpg' on the path, this changes that to 'gpg2' instead -->
    <gpg.executable>gpg2</gpg.executable>
    <!-- <gpg.passphrase>secret-passphrase-here</gpg.passphrase> -->
  </properties>
</profile>

Note the commented out property. There is a step during the Maven release perform goal where the gpg-plugin runs that will sign the artifacts generated for the module(s). If your key is encrypted with a passphrase, a prompt will appear. stdin for this prompt isn’t reachable, so you have no way to enter your passphrase, and the perform goal will fail.

You have 2 choices here:

Set gpg.useagent to false, and keep a plaintext copy of your passphrase in this file.
Set gpg.useagent to true, and remember to interact with the gpg-agent before running the Maven Release so that it can cache your passphrase.

Use your best judgment on what you are comfortable doing with your passphrase.

If you are using gpg.useagent=true

Before you run the Maven release, you need to interact with gpg-agent to get your passphrase cached. I’ve created a small gist with the steps.

Create some temp file.
gpg2 -ab that-tempfile
You’ll be prompted by gpg for your passphrase. Upon success, you’ll see another file with .asc extension next to the tempfile.

Now the gpg-agent in this terminal session has your passphrase. If you repeat the steps, you’ll note you don’t get asked again for the passphrase! Now run the Maven release process (mvn release:prepare, mvn release:perform) in the same terminal session.

Release Management

09 Oct 2015

The following are the slides from a talk I am giving on Friday, October 9, 2015:

Release Management

The premise behind the talk is to describe why, and to some extent how, to include “Software Releases” within your Continuous Delivery pipeline. A Software Release isn’t the same thing as “deploying/shipping/rolling out” to any particular environment; those verbs are the action you perform on the deployable thing that the release creates.

I think releases are an important step for a few reasons:

Releases are really easy to make and cost nothing.
Releases have an identifier that expresses something meaningful to people beyond your developers.
Releases make tags in your version control.
Releases make deployable things.
Releases are immutable.

All of these things give you clarity:

What is in production right now?
What was in production yesterday?
What is the significance of today’s release?

Releases also give you a repeatable way to deploy any milestone of your project; going back to any particular state is a lot easier, and doesn’t depend on a single individual or a data recovery system.

A lot of this is covered in the presentation and the notes. What’s not shown in the slide deck is the tour of my team’s chosen tools to perform release management.