It's Alive: Tips and Tricks on Surviving Your Project Go-Live

"It's Alive!": My Upcoming Presentation at Ottawa SharePoint Meetup Group on April 30

Please join me at the Ottawa SharePoint Meetup Group's April 30th meetup, where I will be presenting "It's Alive!".

You've got to get a million small details right when launching a project, and the pressure is enormous. I will present a list of tips and tricks to assist you with project Go Live, whether you are just starting your project planning, or up against a scary deadline!

We meet at Patty Boland's pub at 101 Clarence in the Ottawa Byward Market where you can enjoy the presentation with good food, drink, and company! If you are planning to attend, please RSVP here so we can arrange enough seats.

Managed Metadata Term Set Tools

One of the great new features of SharePoint 2010 is the new Managed Metadata service, which allows you to centrally manage your metadata. You can setup hierarchical terms in a variety of languages, and delegate the administration responsibility to end users such as Information Management staff or Records Managers.

Although it is easy enough to add individual terms, there are a variety of great tools that make it easy to create and upload entire term sets (the largest we have migrated so far is Medical Subject or MeSH data with about 16,000 hierarchical medical terms).

Here are some of the tools we use at StoneShare:

Excel Template with Macro

Wictor Wilen has created an Excel Template with a Macro to make it easy to populate SharePoint 2010 Term Sets.

I have uploaded the file to the Agora Development site under Shared Documents > Development Tools: https://agora.stoneshare.com/dev/Shared%20Documents/Development%20Tools/TermStoreCreator.xltm

The instructions are located here:

http://www.wictorwilen.se/Post/Create-SharePoint-2010-Managed-Metadata-with-Excel-2010.aspx

Term Set Importer / Exporter

When uploading large term sets (such as the MeSH one we used for a client) you might get timeouts in Central Administration – this is because the out-of-the-box import tool uses a web interface). There is a great (FREE!) tool on CodePlex which uses a little desktop utility, and hence does not time out:

http://termsetimporter.codeplex.com/ 

Pre-Built Term Sets for Sale

Rather than going to the trouble of creating your own term sets, you can also purchase existing term sets from Data Facet. I haven’t yet used these so I can’t speak for them but this could be a good way to quick-start your managed metadata for a particular industry.

http://www.datafacet.com/sharepoint.aspx

One advantage of this approach (apart from the time saved recreating these) is that you will be using the identical term set as other organizations, which may help with data portability. Whether this is your business requirement or not is an open question.

I hope these tools help!

There is a great SharePoint maturity model created by Sadie Van Buren we use to help educate our clients and anticipate their growing use of our products and solutions.

It's easy to use - just examine the particular section you are considering, and depending on the answers to the statements in the section you can determined where the organization fits. Generally organizations have different maturity in different areas of SharePoint.

SharePoint Maturity Model

SharePoint Governance Site Advice

While preparing for my SharePoint Best Practices Conference presentation on centralized taxonomy, I made a sample governance site to help with my demonstrations. Afterwards I received feedback from people asking for concrete examples on how to do this. So here goes:

Steps to Creating the Governance Site

I'm calling this the Governance Site because it's less confusing for the end users, but in SharePoint terms it's really a site collection. Here are some suggestions on how to set it up:

1. Use a dedicated site collection. By making a dedicated site collection you retain maximum flexibility. You can lock down the site collection security for sensitive governance documentation or make most content publicly available. You can deploy custom content types, site templates, and list templates, as well as manage features separately from the rest of your portal. You can manage Information Management policies and retention policies more easily. You also retain the ability to move the governance portal to other content databases, and can make complete backups of all your governance content.

2. Create a new managed path for the governance site in Central Administration. Under Managed Paths, create an explicit inclusion for the keyword "Governance".

2. Now create the governance site collection. With the Managed Path you just created, users will probably navigate to it using the URL http://[your portal address]/governance . The Site Template that you will probably use when creating this site collection is either the Blank Template,  the Team Template, the Document Center, or (probably best of all) the Collaborative Portal - because this will give you news and a site directory right out of the box.

3. Add a new global navigation link on your governance portal, pointing to the home portal. This allows your users to navigate back to the home portal from the governance site collection.

4. Add a new global navigation link on your home portal, pointing to the new governance site. This allows your users to navigate to the governance site right from the home portal.

5. Ensure your Shared Service Provider search settings will crawl the governance site collection, although this should be happening by default.

6. Modify the governance site collection to suit your business. See below for tips on what to include / layout.

7. Keep your content types, custom site columns, workflows, Master Pages, and other SharePoint resources in this governance site. Centralizing them here effectively means they become your "Gold Standard" for any portal modifications. Your administrators should refer to them when making changes elsewhere. Your challenge is to ensure that work done elsewhere is consistent with the governance site.

Governance Site Layout

I'm including screenshots of the work I did. This is intended to help as a guideline.

The Home Page

Obviously this first page has to make it crystal clear to users what the governance site is for, and how to use it.

News and Announcements are important to indicate progress on the portal. A little introduction by way of a content editor web part would be useful to. Whatever you do, make sure this stuff is up-to-date - if you don't keep the governance site content fresh and useful, users will wonder what hope there is for the real portal.

I suggest putting a Roadmap on the front of it to show them where SharePoint has come from and where it is going in your organization. That way they understand this is a process and successes have already been achieved.

I like Roadmaps because people get very excited about the SharePoint possibilities. Having a Roadmap allows the governance team to take requests and good ideas from the business, organize them into little projects, and slot them into a proposed timeline. This encourages the business to see the SharePoint portal as an evolving, enabling process that they are involved in. It also acknowledges the reality of limited technical resources, time, and budget, and the need to make hard decisions about what will be done, and when. Best of all, it's quite transparent!

This road map is simply a Project Tasks List called "Portal roadmap", created on the root of the site collection. Then I added the task list as a web part in Gantt view.

Subsites

I would suggest including at least 5 subsites. These are:

1. Communication

2. Policies and Procedures

3. Support

4. Training

5. Taxonomy

These are the five main things your governance team will probably be working with. You can use any template you like, although in this case I used Team Sites. The Fantastic 40 might be useful for some of these sites - especially the Support or Training sites.

Communication

The Communication site is simply a central place to spread the word about the SharePoint portal. It could be very top-down, push communication via news items or updated content. More collaborative communication would probably involve wikis, blogs, or discussion threads. If you have an existing communication process or software you could link to it from here. Tying the SharePoint portal into your existing processes and content is an important way to make SharePoint seem less threatening, more empowering, and more inevitable :)

Not shown here is a feedback mechanism. This could be a discussion thread or embedded poll that asks users to comment on the governance site and suggest ways to improve it.

Policies and Procedures

The Policies and Procedures could be a Document Center. It is simply a central place for users to find all governance documents. You should store them here and then link to them from all the other sites, using the Link To A Document content type that is available from SharePoint out of the box. This will cut down on redundancy.

Support

The Support site should have a support matrix, showing users what levels of support are available for various sites, applications, and problems. This could range from self-help to IT support. There should be contact information for help desks and IT staff if available.

This is a good place to store videos and cheat sheets for users to help themselves. SharePoint add-ons such as the SharePoint Learning Kit could go well here. Effort place here will cut down dramatically on support costs!

Training

Training could also use the SharePoint Learning Kit, and could be a good place to identify what courses are available (internally and externally) and request training. I worked with a financial client who would not let users onto the SharePoint portal until they'd passed a small online course. Each manager had a KPI and was responsible for ensuring their staff took the course, passed it, and followed its teachings when using the portal. This helped lower support costs, the staff were very satisfied with the portal, and they used it more.

Taxonomy

The Taxonomy site is a place to document the metadata and site structure you will be using to organize the home portal. This site will be more technical but is an important resource for your administrators, developers, and content authors.

Document your content types and site columns, when and where they should be used, and explain how SharePoint uses them (advanced search, property searches, content query web parts).

If you have an organization Thesaurus, document or link to it here.

This is what it could look like:

 

Governance Site Functionality

There's a lot of native SharePoint functionality that you might wish to use on your governance site collection.

Auditing on the site collection: You may want to know who is looking at what.

Workflow: The Governance site is an excellent candidate for light-weight Out-of-Box or custom workflow. Workflow can help triage support and training requests, and handle publishing approval for governance policies and procedures.

Search Scopes and Best Bets: If ever there was a place to serve up highly targeted search results, the governance site is it!

Search use metrics. You have to turn this on in your Shared Service Provider(s). Pay very close attention to what people are searching for and finding (or not finding) on the governance site and use this to modify your governance approach.

Audience Targeting: Consider audience targeting content on the governance site to prevent information overload. Broadly speaking your governance site audiences might include:

1) Governance Team members

2) IT Operations, Farm Administrators and /or Site Collection Administrators

4) Site Owners (Probably business owners and power users)

5) Content Authors and Champions

6) Everyone Else

Key Performance Indicators - if you have the Enterprise license of MOSS then Key Performance Indicator functionality can be useful to give people a heads up of roughly how portal adoption is going. One thing you can do is create a custom list with some tracking metrics and then put a KPI on that list. You can update the list manually each week to show the latest usage trends.

Metrics are very important as they can indicate whether your governance effort is leading to success.

Information Management Policies - especially if you are applying records management practices to the governance content.

A Starting Point

I really hope that helps. If you do create a governance site, try to keep it up-to-date and make it the central place for people to visit. I would love to hear your comments on what you did, what works for you, what tips you have and what add-ons you found to help you manage your SharePoint portals.

Good luck in your governance efforts - it will definitely be time well spent!

P.S.: Sorry this post is a couple of weeks late, Julie :)

I'm baaaaaack! Back from Oceania and freshly installed in my native city of Ottawa, Canada. The last couple of years have been a happy blur. I'll definitely miss Oz and the great people at Dimension Data Australia but it's nice to be in the same time zone as most of my friends and family.

As I mentioned in my previous post, I am working on some new projects.
One of these is collaborating with echoTechnology as their new Technical Product Manager to help develop and promote their tool, echo for SharePoint. This is like a Swiss-army knife utility suite that makes changing SharePoint easy. It's a product I've had my eye on for over a year, ever since I first interviewed the echo guys in these posts: echo demo and interview.

The reason I'm so keen on the tool, and why I agreed to work with them, is that after two years of SharePoint consulting I have yet to find a suite of tools that makes it so easy to manage SharePoint configurations and settings. Trying to migrate my solutions from Dev to Test to Prod has been an exercise in frustration. Keeping the state of a farm consistent when multiple team mates worked in parallel, pushing out changes to more than one site, or sharing the deltas with colleagues who are working in their own virtual machines has required a relentless search for scripts, code snippets, and any means possible to gain a little development consistency and control.

Part of my brief with echoTechnology is to work with the community to help develop best practices on SharePoint governance and change management. This is an area dear to my heart, and the focus of many of my workshops and engagements over the last few years. I've devoted a fair bit of space on this blog to those kinds of issues and that will continue.

Whenever possible I plan to release white papers, site templates, and snippets of code on this blog. I hope they will be useful to anyone planning to govern their SharePoint implementation - which should be EVERYONE!

Data In, Data Out

An important emerging web development initiative concerns "data portability", or how openly and transparently a website or application consumes and provides data. In a world of mashups, web services, and walled gardens like Facebook or MySpace, it is becoming an increasing concern for a variety of reasons.

To begin with, users want to know that the information they enter into a website is theirs to manage, share, and remove over time.  This matters for convenience (so they don't have to reenter it all the time) but also from a privacy perspective.

A famous recent example saw Robert Scoble temporarily banned from Facebook for breaking its terms of use by running a script to access his social graph (contacts). This focused a great deal of attention and debate on who "owns" this sort of information - the person entering the data, the website that hosts it, or the individuals whose information is being stored.

Developers also care about this as we are all tired of reinventing the  wheel every time we want to share data with another application. A major consequence of the recent wave of web development trends is the increasing importance of application integration and data integration standards. These days, "No Web App Is An Island".

To help address these needs, a community is coalescing under the banner of the DataPortability Group. Their website is located at http://www.dataportability.org/ and discusses the issues in detail. They define portability as

both physically moving data or simply porting the context in which the data is used

Their effort involves identifying and evangelizing existing data portability standards, rather than creating new ones. They also hope to encourage a trust framework that will benefit vendors and consumers.

The rapid adoption of this initiative shows how quickly things move in the IT world, if proof were needed. The project was first founded on October 11, 2007, and is already gaining a great deal of steam. The website has an interesting timeline showing how quickly support is building: http://groups.google.com/group/dataportability-public/web/buzz . 

Some recent events of note included Google, Plaxo, LinkedIn, Twitter, Flickr, Six Apart and Facebook joining the workgroup; Google, Yahoo, IBM, Microsoft, and VeriSign joining the OpenID Foundation board; and MySpace launching its Open Developer Platform.

Obviously most of the activity is driven by the Social websites as they have the most to win / lose. Issues of personal privacy and trust are crucial to their continued popularity (the Facebook Beacon PR disaster is a prime example of this). 

Nonetheless, I imagine within a few years every major Software vendor will have the DataPortability-compliant tickbox in their sales material, or their shareholders will demand to know why not. Case in point: Blogger, owned by Google, now supports OpenIDs on its blogs (such as this one), and Yahoo users can use their Yahoo! accounts as OpenIDs.

Interestingly enough, some of the standards the DataPortability group advocates are the ones that might lead to the long-envisioned semantic web - namely microformats, which can potentially add machine-readable "context" to data.

It's an interesting space to explore, and I plan to blog about it as I learn more.

Shaun Leisegang, K2 principal technical specialist, gave a webcast just now about how K2 blackpearl integrates with several Microsoft technologies. He showed how K2 workflows can be built and hosted in MOSS 2007, in Visual Studio, and in Visio. Some of the things I learned (in no particular order):

• Environment variables make it easy to transition workflows between environments without modifying any code;
• The updated K2 Tasks Web Part has a batch functionality allowing a user to select and approve or disapprove multiple tasks at once. The web part also has a new quick search that helps filter the lists of tasks, and exposes task actions right on the web part for redirecting (reassigning) tasks, delegating them, or performing some other action.
• The Visio integration allows a user to import existing K2 workflows into Visio...Once a workflow has been chosen from the file system, it pulls in all the workflow elements into the Visio environment. At this point those elements are "orphan activities" but a user can select an object on the Visio page, and assign one of the orphan activities to it, create a new activity, or make it the "start" activity. This process is essentially a mapping between the objects on the Visio pages and the K2 workflows.

Shaun showed how easy it was to add things like email disclaimers to workflow templates. Since K2 is built on top of Windows Workflow Foundation (WF) each of the workflow activities can have custom code.

Shaun also referred to two K2 templates for SharePoint user and document management which help with provisioning and permissions, although these weren't demonstrated. At the end of the presentation one of the questions was about these: they were described as
"the SharePoint Document Event template to handle the movement and manipulation of documents and the SharePoint Users Manager event template to handle MOSS security permissions". Both of these sound quite useful as they are frequently requested by clients.

Another attendee asked about Oracle integration. Shaun stated that templates to connect to Oracle have been developed and K2 "also supports the Microsoft BizTalk 2006...adapters which support Oracle".

K2 has announced a release date for blackpearl! The date will be August 6. From the press release:

K2 blackpearl gives business and technical users the ability to collaboratively design, assemble, execute, monitor and optimize dynamic business applications. K2 blackpearl includes business process management capabilities. Although blackpearl's capabilities far exceed those of K2's previous offering, K2.net
2003, existing customers can upgrade easily.

I'm not sure if pricing and licensing information is currently available somewhere or will be published on their website; as I recall for the previous version you had to inquire about it.

K2 will present information and product demos this week at the Microsoft Worldwide Partner Conference 2007 in Denver, Colorado.

You can view the full press release here.

After K2's BlackPearl demonstration a few weeks ago, I had a few questions. I contacted Josh Swihart at K2 and he and Anthony Petro, the Technical Product Manager, replied to me:

Question: I know the BlackPearl Report Builder uses Reporting Services – can these reports be exported and even hosted on a distinct Reporting Services server or hosted in SharePoint’s integrated Report Centre? The reason I ask is that a lot of Microsoft products - MOSS, Dynamics CRM - all use Reporting Services, so from a server topology perspective it would be great to have them all publish to the same RS server and manage things a little more centrally. Josh and Anthony: Yes, when you install BlackPearl you must provide information on which Reporting Services server you want to connect to. The Report Designer will use this server when you “publish” a report to RS. Once the report is in RS, you can use standard UI in RS, or compatible apps like SharePoint RS web parts, to view these reports.

Question: I wondered if the autogeneration of InfoPath forms using that Forms Designer item that Adriaan demonstrated would be available by Release-To-Market or shortly thereafter. This would be a nice-to-have for InfoPath 2007 development in conjunction with MOSS.
Josh and Anthony: The SharePoint integration wizards can use either ASP.NET or InfoPath forms server forms. Both technologies will have default forms that are installed with BlackPearl to be used for Instantiation, Association and Task Form. Note that there is more info on this in the SharePoint whitepaper. Additionally, we have added the Forms Generation Client Event which will allow you to generate ASP.net forms automatically for Process, XML and SmartObject data in the process. Post-RTM we will expand the forms generation client even to support autogeneration of InfoPath forms.

Question: I read this in one of the K2 Black Pearl pdf documents:

“K2 “BlackPearl” process information is availableto MOSS users through
integration with the MOSS Business Data Catalog (BDC). All process information is
searchable (security is respected) and can be exposed through list columns and
BDC web parts.”

Does this mean that for any custom workflow I create in K2, a BDC application schema is automatically generated for me to upload into the BDC Catalog? Or do I still have to create a schema for this by hand? It mentions “process information” - does that mean just the time it took the workflow to execute, the name of the originator, etc, or does it also expose SmartObject data and the like so that I can pull those into SharePoint via the BDC?
Josh and Anthony: We provide the ability to surface any SmartObject data through BDC - including process and activity data configured as SmartObjects. We have created a K2 Admin tab in SharePoint Central Administration to make it easy to identify which SmartObjects, and optionally their associations, should be exposed in BDC web parts and/or search. The BDC application metadata for connecting to the SmartObjects is automatically generated and configured as result of the actions on the admin page.

Thanks Josh and Anthony for taking the time to respond!I'll also mention again that there is a new community forum called K2 Underground for exactly these kinds of queries. There is also some documentation available on K2's blackpearl site which describes some of the new features. Plus there's a beta to download and play with!

K2 Underground is here: http://k2underground.com/. K2 Blackpearl website is here: http://www.k2.net/bp. You can send an email to beta2@k2.net to get your hands on the beta 2.

K2 has a new community-driven website, K2 Underground, which replaces the existing forum.k2workflow.com site. The new community area allows registered users to participate in forums, blogs, groups, and file shares, although unregistered users can browse the site. It's a good idea and looks well built; it'll be interesting to see if the K2 community coalesces around it.

You can access K2 Underground here.

Yesterday I saw a great webcast about K2 [Blackpearl] given by Adriaan Van Wyk, CEO of SourceCode Technology Holdings. For those of you who haven’t tried it, K2 is a Business Process Management (BPM) or “workflow engine”. BlackPearl is pretty exciting stuff because it is built directly on Windows Workflow (WF), Microsoft’s new workflow framework.
K2 has just released a public Beta 2 version of their engine. Beta participants will receive a 25% discount on all Blackpearl software if they order it before 30th June 2007 (but licencing info won’t be made available for a few more days!). RTM is expected within a few months.
Key points brought up from the webcast:

• BlackPearl is heavily integrated with the Office 2007 stack, including SharePoint. K2 workflows are hosted in SharePoint libraries just like the out-of-the box MOSS workflows are; the appropriate workflow actions are easily available from the Actions menu; workflow tasks appear in Office 2007; and so on.
• Workflow development “scales” according to technical ability and need. A Business Analyst can draw up a general business process using Visio and deploy to SharePoint. A developer can import that workflow into Visual Studio, add more complex logic, and then re-export the changes.
• Related to that last point….the K2 Studio is now fully integrated into Visual Studio – so no mucking around in a separate IDE!
• All K2 BlackPearl workflows are xoml files (the WF format).
• All workflow actions are fully surfaceable using an API, which means you can easily incorporate them into your other applications.
• K2 provides the source code for its templates, allowing you to poke around and customize as required (although whether they will support any template changes you make remains to be seen!).
• Exporting K2 2003 workflows to a target server was a bit difficult. Now you can publish your workflow as a CAB file, which combined with an XML configuration file for each environment means you can deploy K2 workflows with no K2 software installations required at all on the target servers! This will really please server administrators. During the webcast Adriaan demonstrated how you can extract a K2 workflow CAB file using a command line on the server.

Some other nice features:

• There is a Forms Generator tool to autogenerate .aspx pages from K2 workflows. Autogenerating InfoPath 2007 Forms will not be possible at RTM but hopefully shortly thereafter.
• Workflow process information can be surfaced in SharePoint using the Business Data Catalog.
• Smart Objects allow you to keep your BPM data outside of K2. This is a great feature because you can reference the Smart Object’s unique ID in the workflow, but update the rest of the SmartObject’s properties over time, and all workflows using that SmartObject will still work. This simplifies maintenance of data structures.
• Reports are generated using Report Definition Language, which means you can create and customize them using Reporting Services technology.
• A Context Grid allows K2 to make decisions based on the runtime environment, such as rerouting tasks or messages based on a user’s availability (perhaps determined by Live Communications Server).
• Black Point is an AJAX designer which is hosted in MOSS. This component allows business users to generate and modify workflows without even leaving their browser window.

Workflow is a vital component of the SharePoint story – enabling customizable, real-value business solutions in a reasonable timeframe. You can already develop workflow using Visual Studio or SharePoint Designer, but a sophisticated product like K2 really takes much of the pain away, allowing you to focus on the business requirements. K2 continues the trend towards declarative software development.

With SharePoint 2007 and workflow forming a solid platform, I think we’ll soon see a new kind of Extreme Programming: one developer and one business analyst paired together at a computer, happily rolling out business solutions.