Process improvement by shifting left and reorganizing

Introduction

 Throughout my career, I have always had a great vision in identifying areas for process improvement and maximizing work. As enterprises adopt lean agile strategies in accomplishing work the one area, I see struggle with; especially in Information Technology where we are trained to solve problems using technology, is procedural improvement in how workflows into IT and how IT is organized to solve work. This blog article will discuss areas where IT can identify improvement in process and recommendations around how to organize to improve work input and business value.

This blog will introduce a few concepts that I will share links to for you to learn more about them. This will be part 1 of several blogs that I want to discuss on this topic. 

"Shift-Left"

The concept of shifting left is how tasks can be accomplished quicker by introducing it sooner in the process to reduce waiting, improve quality, and finish faster. Most of you might already be familiar with the concept and do not realize you experienced it! Good real-world non-technical examples of shifting left are ATM machines and self-checkout stations at popular supermarket stores. 

You are probably thinking "Wait what? How is that shifting left?" Great question! Think about the process of obtaining money from the bank. Traditionally you walk into the bank, wait in line, fill out a withdrawal slip (or write yourself a check), hand it over to the teller, teller enters account information, checks ID, withdrawals cash, and hands over the cash. That is a pretty lengthy process to obtain your money. The ATM eliminates a good portion of the process, and more importantly, shifts the process of obtaining your money to you and allows you to obtain your money faster and more securely. 

In Information Technology the concept is the same and to learn more here are two examples of how it can apply to Software testing and end-user support. 

Agile Manager

This concept could have a blog just dedicated to it as it is a broad topic that covers various aspects of organizational management combining the theories of Agile software development, humanist organization, and parts of system theory to organizational management. To describe it in easy-to-understand terms an Agile manager is not a manager but a leader that empowers people to contribute to the decision process, ensures they are given the tools to be successful, and most importantly always identifying areas for improvement to the organization to ensure the success of the overall team. Essentially, an influencer more than a manager. 

Two good articles are Agile manager are Accenture What Makes an Effective Agile Manager and An Organization Development(OD) Approach to Agile Adoption.

The Repair Depot

In the early 90's I worked for a large PC manufacturer fixing computers in their warranty repair depot. The repair process was a linear process: 

• work order is scanned by tech and computer is now assigned to the technician
• technician diagnoses the problem in comparison to the customers described issue
• If hardware parts are ordered against the workorder from the parts department and delivered by a parts runner
• If software and requiring factory reimage the computer is brought to the imaging station where it is released to the station waiting for image
• Upon repair the computer is sent down a conveyor to a Quality Assurance technician who does Quality checks and tests to validate the repair
• If the repair is not satisfactory it is sent back to the technician for rework
• If repair is satisfactory it is sent to the shipping department, where it is shipped back to the customer

On the surface, the way work was performed is very logical. There is input, analysis, repair, and QA. However, there is a lot of wait time and testing is performed at the end of the process. This waits time impacts technician productivity and potentially increases the time the computer gets back to the customer, reducing overall customer satisfaction. This is if QA passes the repair. If it fails this process could be repeated several times! 

The inefficiency of "wait"

In this process waiting equal productivity lost and poor customer experience. Since technicians are encouraged to repair at least 1 computer per hour technicians would often "cherry pick" work orders based on ease of repair, known part availability, and/or problem stacking [the process of identifying a common problem like imaging and doing multiple tasks with a single process such as loading up the image station with several computers to image to meet repair obligation by end of day].  This culture led to not all work orders being complete as somewhere left and aged past the SLA resulting in non-assigned work orders to stack up requiring weekend work and/or overtime costing the manufacture of unnecessary costs and even poorer customer experience due to delays. 

"Shifting-left" to reduce wait!

I had a great opportunity to join another PC manufacturer as an influencer in how the repair process could be improved. I was hired to solve the problem around technician throughput; how to encourage more output from each technician as well as improve overall time of repair and get the computer to the customer faster while not increasing return repair. 

To solve the problem, I recommend shifting as much of the process to the technician to streamline the repair process and reduce the wait. 

• Parts ordering: I identified the 10 most common hardware issues that customers sent their computers in and created a new parts location called "floor" and put these parts next to the technician. This eliminated the need for part runners reducing the time to obtain a part from potential 30 minutes to less than a minute while ensuring asset tracking and not impacting inventory management. 
  
  By also having common parts available reduced technicians from wanting to "Cherry pick" work orders improving repair time and customer wait time for repair. 
  
  
• Eliminating image stations: An image server was setup on the network allowing for images to stream from a central server and the technicians were given enough space in their work area to be able to image multiple computers at a time while also repairing other computers. This eliminated the bottleneck of having to wait on the image station and improved overall repair time and customer wait time for repair
  
  
• Testing at the bench: Technicians were given the QA tests and were required to perform tests after the repair eliminating the need for QA to perform tests. By doing tests at the bench validated the repair and allowed for faster resolution if there were still more errors. This enabled QA to have faster tests validation of repair eliminating the bottleneck to sending the computer to shipping to return to the customer
  
  

These 3 changes reduced the wait time, improved workflow, and improved customer satisfaction by having the repair performed quicker. Not only did this improve the process but technical satisfaction was higher at my new place of employment than at my previous place and our productivity was far better with very little overtime required. Our customer satisfaction rate was around 98%, improving from 90% prior to the change. 

In the next part of this blog series, I'll use this example to discuss agile management and how these concepts can improve service delivery for Enterprise IT. 

Integrating Unix into Active Directory while maintaining UNIX style security - 2018 Update

Integrating Unix into Active Directory while maintaining UNIX style security - 2018 update

In 2010 I had the unique challenge to research how to integrate Unix and Linux into Active Directory. While new tools such as Centrify, Quest, and Likewise (now BeyondTrust) had clients you could install on your Unix clients they came a heavy cost; and in most cases, was a NIS layer (at the time) over your AD to enable UID/GID mappings. 

I wanted to have the ability to use OpenSource tools and enable the ability to have auto assigned UID to user and groups so UNIX style security could be used on resources such as NFS. The solution I came up with borrowed from Apples Magic Triangle configuration using Apple Open Directory and Active Directory.  The exception was I leveraged OpenLDAP as the UID/GID database while keeping User and Group objects in Active Directory. This enabled Linux to use the opensource SAMBA client to authenticate to Active Directory while using OpenLDAP for the corresponding UID/GID for the user and groups objects using SAMBA algorithm that generates the UID/GID using the Active Directory Object SID.

This solution works well for the time, but there are two other solutions that we can leverage that is far easier to configure and provide a more native experience. 

Option 1 SSSD

The System Security Services Daemon (SSSD) supports Active Directory integration. Like previous SAMBA clients a UID/GID is auto-generated from the Object SID, but it's only system wide. This works great if you are using a server and you want to secure only to this server, but if you are leveraging shared services across multiple servers, we need a solution to have a central UID/GID database that is unique within the Directory. 

My first thought was to do what I did before and leverage Python Active Directory module on a dedicated Linux server and have it generate objects UID using SSSD query; but that limited me to the SID and increased complexity account creation. Also, for existing accounts I would have to have a script that queried all objects then update the object to retrofit my environment. 

My biggest concern outside of increasing complexity was ensuring a unique UID/GID in a multi domain Active Directory Forest. Using the SID might not be unique; especially when dealing with other trusted domains. 

In Active Directory the Object GUID is unique to its forest and unique forest wide. I wanted to create an algorithm using the GUID to create a unique UID/GID. While researching ways to convert the 128-bit integer to 32-bit UID I came across a blog where an engineer reversed engineered Apple's client algorithm that does just that: https://themacwrangler.wordpress.com/2016/11/29/reversing-the-ad-plugin-uid-algorithm/

Using the script examples above I created a PowerShell script when objects are created a UID is updated in the LDAP field for that object. By doing this not only will our clients that are configured to use SSSD for Active Directory authentication - with SSSD using Active Directory to obtain the UID/GID - native clients that are using LDAP and/or Kerberos (like an Isilon NFS server); and Windows clients with NFS enabled, can use the UID of the object to grant access allowing Unix style security on Unix based services. 

After the object is created the following needs to be performed to generate the UID of the Object and then an update to the object can be performed and the proper LDAP attribute can be updated (UID for user and GID for Groups)

# Change GUID to UID

 $ConvertGUIDToUID = New-Object guid(,($object) | %{[Convert]::ToInt64(($_.guid.split('-')[0]),16) -band '2147483647'} 

# update user uidNumber 

Set-ADObject -Identity $ObjectDN -Add @{uidNumber=$ConvertGUIDToUID} -Server $dc -Credential $Cred

# update group gidNumber 

Set-ADObject -Identity $ObjectDN -Add @{gidNumber=$ConvertGUIDToUID} -Server $dc -Credential $Cred  

Option 2 OpenLDAP Pass-through Authentication

Like my previous blog this option leverages OpenLDAP, but instead of using OpenLDAP to be an IDMAP database to store UID/GID you leverage OpenLDAP as your main authentication and leveraging SSL trust between OpenLDAP and Active Directory the authentication is forwarded to AD from LDAP. 

The benefit to this is you don't have to rely on SSSD Samba based client to authenticate and use native LDAP client with UNIX. Since we are using native LDAP, this allows for broader client support; especially older Unix systems that might have issues talking to Active Directory as an LDAP provider. 

In my opinion the use case I prefer to use this in is for clients that might reside in the DMZ; for example, an LDAP based service that doesn't support SAML or OpenID, and provides SSO for clients. Only SSL needs to be allowed between OpenLDAP and Active Directory. 

For UID and GID we still need to generate a unique ID for the object. We can mirror the object in LDAP (can cause issue) and have a domain specific ID mapping like traditional LDAP or we can auto-generate as we did in Option 2 and allow the pass-through authentication to retrieve the UID and GID. 

In my opinion Option 1 is preferred unless you have a specific use case that requires this option. 

https://blogs.msdn.microsoft.com/alextch/2012/04/25/configuring-openldap-pass-through-authentication-to-active-directory/

Using Active Directory groups to control access to Linux services

Using Active Directory groups to control access to Linux services

Integrating Unix into Active Directory while maintaining UNIX style security

Integrating Unix into Active Directory while maintaining UNIX style security: "Integrating Unix into Active Directory while maintaining UNIX style security"