TechEd 2008 Session Notes: MOSS Administrative Architecture, Deployment, and Operations (Part 2 of 2)

This was the second in a two-part series given by Joel Oleson, Shane Young, and Mike Watson. The session was highly technical in nature and consisted of some very valuable and very specific do's and don'ts for advanced SharePoint administration.

Tip: Add command line environment variable shortcut pointing to the Office 12 Hive location on your server, since you will have to go there a lot

Site Collection & Content DB Management

Tip: Put similar sites together (based on content sizes and # of users). This can help manage functionality and configurations.

Best Practice: Separate My Sites from portals so they can be moved and managed separately.

Best Practice: Large site collections should be dedicated to a single database.

Tip: Application pools:

1. 32 bit – can handle up to 1300 mb per app pool before recycle
2. 64 bit - up to 6 gigs per app pool

No best practice for how many Content Databases is the right number. This is more of a management issue for the business's IT staff.

Tip: Don’t put too much content into one site collection. Try to keep DB sizes under 100 gig – rough rule of thumb

Best Practice: Database Maintenance Plan: In SQL Server, do not use default growth plan, don’t autogrow in production, and plan sizes ahead of time

Disaster Recovery can be done via Stsadm, SQL Backups, Log shipping, remote snapshots

Best Practice: When Moving Config databases: use preparetomove, then detach and reattach (otherwise you get GUID problems!)

Customization and Development

You have to have a policy for how this happens. It HAS to use features and solutions or it doesn’t make it on the server!

Best Practice: Use Code Access Security (CAS) when coding

Best Practice: Test for memory performance and disposal of objects

Best Practice: Test adding and removing solution to the farm

Tip: To see which solutions are running – go to admin/solutionstatus.aspx

Best Practice: Deploy Solutions off peak hours, because all deployments will recycle the app pool which will dump user sessions. Also be present so you can test it!

Defaults Have Faults

Server:

1. Default log and app locations are not great
2. Basic setup is NOT recommended since it installs SQL Express
3. Having a different Application pool for every site collection is too much
4. No backup is scheduled by default

Site Collections:

1. No quotas, no expiration, no auditing
2. No versioning, no checkout
3. No approval or workflow required
4. SharePoint Designer allowed for all admins and designers

Database:

1. Max # of site collections is too high
2. DB file location and growth plan are wrong

TechEd 2008 Session Notes: Microsoft IT Best Practices

Microsoft IT ran several sessions on the topic of SharePoint governance. The ones I attended were Microsoft IT: Building Applications and Reusable Code on SharePoint, and Microsoft IT: How Microsoft Plans, Governs, and Operates Custom Applications and Portals within Microsoft. Both were hosted by David Johnson and Sean Squires. David and Sean did a great job sharing their practical experience and knowledge, and cultivating an open dialogue between themselves and the audience.

It was quite fascinating seeing how Microsoft "eats its own dogfood". However the resources they bring to bear make following their example a little problematic.  Microsoft IT use literally dozens of SharePoint farms, which from a licensing perspective alone would be prohibitive for any other organization.

SharePoint Structure

MS has several MOSS environments and many farms:

Vanilla SharePoint farms: staff can collaborate and use SharePoint Designer to tweak, but no custom code. Also includes My Sites. Most people use this.

Dedicated farms: Custom code deployed, no ability to use SharePoint Designer, supported by SLAs. They are currently trialing Enterprise Agreement-style process for helping Microsoft business units renew their portal use. They have to adhere to these agreed-upon standards or they can't use a dedicated farm.

Platinum – 7 major portals, specific customization and LOB applications, plus much more governance.

How They Do It

Created a diagram of information landscape and "pillars" - a catalog of existing code which allows reuse across the organization

Solution Catalog – list of custom solutions and samples

TIP: Don’t use MOSS as a relational database!

Best Practice: Standard application architecture - design for reusable components and in layers

Best Practice: All external UI labels and messages are stored in resource files (.resx) and SharePoint lists - this allows rapid UI changes without recompiling / redeploying. Configuration done via lists

TIP: XSL is critical to parse and present XML. Related to this, any custom UI uses token-based HTML templates

TIP: Make heavy use of Content Query webpart. There is a newer version of this with some changes to make it easier to modify

Security – typically use AD groups rather than SharePoint group. This is consistent with Microsoft's existing AD groups. Also when using SharePoint Groups, adding and removing users automatically causes a full recrawl. MSIT do create separate AD groups for publishers and creators

Runaway Successes

Vanilla SharePoint is basically a utility environment – employees can run wild subject only to quotas

MSIT – didn’t want to over-govern due to their highly technical staff

However this sometimes leads to “Runaway success”/runaway portals

Cleaning up a runaway portal becomes a matter of business negotiation

They are building new reports to at least assess what’s going on

To fix portal sprawl they are trying to migrate them back to vanilla environments, using scripts to do cleanup.

SharePoint Designer cleanup is much harder

Best Practice: You should organize portal on employee needs rather than org chart which often changes

Dogfooding

Microsoft has to upgrade early in the development cycle of the next generation of SharePoint, so vanilla environments help with migration

Dogfooding – Microsoft IT adopt 3 “9”’s for availability (99.9%) and no more than that, because they adopt Service Packs and new versions early

Best Practices for SharePoint Governance

MSIT are still trying to arrive at MOSS best practices themselves – they feel they are still in their infancy in terms of governance

Biggest Governance and IA challenge is breaking down organizational boundaries

Create a central place for governance

Create a central area for task checklists, best practices, and criteria for development

Best Practice: Create a “Playbook” of solution / configuration options – so business groups can look at this before they engage. This helps align them to existing solutions and the way SharePoint works.

Create scenarios – during consulting phase – to try to prove value of out of box functionality.

TE2008 Session Notes: Planning and Implementing Global MOSS 2007 Deployments

The next session I attended was given by Doron Bar-Caspi, Mike Watson, and Joel Oleson. It gave a great overview of things that need to be considered when implementing SharePoint across a WAN. The session covered the various deployment options, suggested information architecture, and gave tips and best practices to handle LAN performance testing.

Geographically-Dispersed Portal Challenges

• Cost vs Usability
• Bandwidth can be bought but latency is bound to the laws of physics
• User experience when accessing information across the WAN

General Options

1. Central Model
2. Central with Regional Portals Model
3. Distributed Model

Global Scale Search

Search Options:

1. Index centrally for everything. Impacts the WAN due to indexing but this can be throttled on the index server.
2. Index regionally for regional users. Each region hosts its own SSP. Results in two crawls but can be optimized for search

Federated Search:

New option, part of Microsoft's new Search Engine (which uses the SharePoint Search engine).

MOSS can now get results from remote search servers

My Sites

Usually SharePoint services are centralized

TIP: Consider synchronizing user profiles using custom solutions

Distributed model

Local portals handle everything including their own searching

Legal requirements can force this due to restrictions on data storage and use

TIP: WAN accelerators can help improve perceived performance when accessing information from other portals

Optimizing Authentication

NTLM is faster for smaller sessions, but Kerberos is faster for longer sessions.

Kerberos packets are larger, NTLM is easier to support and troubleshoot.

Which Topology to use?

• Know your usage profile
• Know your network
• Test from remote offices

TIP: Be aware that most work is done in My Site or project sites.

Best Practice: Evaluate bandwidth and latency before choosing a global deployment model. Also look at network maps first to decide whether hub and spoke model already exists.

TIP: Bandwidth and latency data was added to the Capacity Planning tool BEFORE the SharePoint model was added; therefore the calculations may not be fully representative of real world scenarios

Best Practice: Use ping to test latencies – using scripts to test at various times to develop a realistic understanding of network performance.

Consider end to end perceived network performance. Don’t assume anything: issues can be browser, computer, router, or network related

TIP: Use Fiddler to help assess network activity from SharePoint.

TIP: If you use the Business Data Catalogue it can be problematic because it makes big calls to SQL – no batching currently

Codeplex has a data population tool and whitepapers on managing geographically-dispersed SharePoint farms.

TE2008 Session Notes: MOSS 2007 Lifecycle Management: Methodologies, Tools, Best Practices

I thought I would upload my raw notes from Tech Ed 2008 Orlando in case they're of use to anyone. Hope these make some sense...

The first session I caught was given by Luis Camara Manoel and Satish Mathew, who are Program Managers at Microsoft. They gave a demo of new reporting tools which help to manage and report on SharePoint usage in an organization. The question they posed was:

How can you govern SharePoint if you don’t know what is out there? Discovery is required.

The emphasis of this session was on the following utilities.

SharePoint Asset Inventory Tool

Agentless utility; crawls network and reports on deployment and topology inventory.

How it works – runs WMI call to see if IIS is running somewhere. Then sees what web apps IIS is running.

Therefore “agentless” and CPU intensive only on the client, not the server (because it makes SOAP calls as if it were one user requesting).

Download: Currently in beta.

Executive Overview: http://technet.microsoft.com/en-us/library/cc295797(TechNet.10).aspx

SharePoint Capacity Planner

Helps see impact of various device utilizations: CPU, Network, Disk IO.

Steps to use: Assess, Recommend ,Simulate, Report. Can export models to Visio and Excel for further work.

Download: http://www.microsoft.com/downloads/details.aspx?FamilyID=DBEE0227-D4F7-48F8-85F0-E71493B2FD87&displaylang=en

Executive Overview: http://technet.microsoft.com/en-us/library/bb961988(TechNet.10).aspx

SharePoint Cross Configurator

Solution Deployment Package to enable / disable features and configuration. Includes audit, expiration policies, content types, master page deployment and enablement. Source code available in CodePlex at http://www.codeplex.com/SPConfigurator.

Leverages stsadm and makecab. To use: provide app.config file to setup.bat.

The tool will run against existing site collections only; new site collections will inherit the default settings. Only works across site collections, not across farms

Download: http://www.codeplex.com/SPConfigurator

Executive Overview: http://blogs.msdn.com/sharepoint/archive/2008/02/06/standardization-with-cross-site-collection-sharepoint-configurator.aspx

SharePoint Monitoring Toolkit

WSS / MOSS/ SQL / IIS Health roll-up

Uses System Center Operations Manager.

Download: http://www.microsoft.com/downloads/details.aspx?FamilyID=E4600FD9-F53D-4DED-88BF-6BB1932794F9&displaylang=en

Executive Overview: http://technet.microsoft.com/en-us/library/bb975149(TechNet.10).aspx

Virtual SharePoint Conference

Last Wednesday I had a chance to participate in a Virtual Trade Show for several SharePoint vendors. This was a nifty little vehicle to demonstrate our products to people all over the world.

The event was organized and hosted by DynTek, a consulting firm, using Live Meeting. They handled all the marketing and arranged for a representative of each vendor to present for up to 15 minutes on the product's capabilities and the value it adds to SharePoint. At the end of each presentation a live poll allowed the attendees to vote on how much a particular tool had captured their interest and notify us if they wanted a follow up demonstration or evaluation.

I ran the echo for SharePoint presentation. The other three vendors were Nintex (presented by Mike Fitzmaurice), ScriptLogic (presenter: Mike Perrault), and BA-Insight (Mark Aschemeyer). As the DynTek invitation put it:

Enhance Workflow & Reporting
Nintex will discuss how you can build complex workflow processes quickly and easily using a web browser interface and provide unprecedented insight into workplace usage, trends and behaviors.
Enhance Permissions
ScriptLogic will discuss how administrators can now backup and restore permissions as well as control the security of SharePoint environments the exact same, clear, intuitive way they would manage the security file servers - all from the same, intuitive console.
BA-Insight Software will discuss how to extend the search capabilities of SharePoint to deliver the most comprehensive and flexible Information Access Platform available in the marketplace today.

Enhance Management
Echo Technology will discuss how to easily migrate and manage change on your SharePoint platform through its entire lifecycle.
Enhance Search

It was a real challenge sticking to the 15 minutes time spot. It meant showing a few slides and setting the scene for why our products added value to SharePoint, and then (if time allowed) running a little demo.

I had a batch of echo Tasks setup ahead of time, so although the time was tight I was able to present live on how we could:

• Deploy 3 custom web parts from one site to another;
• Apply a new Theme and Master Page;
• Migrate files from a file share, apply content type metadata, and insert into a document library;
• Migrate files from a SharePoint 2003 portal into the same document library, again applying metadata as I went;
• Activate a Nintex workflow feature to a site collection; and
• Deploy a Nintex custom approval workflow to a document library

All of these changes were done from a local Development portal to a remote Production portal to demonstrate how easy echo makes that.

Kudos to DynTek for the cool concept and especially to Linda Ford, Julie Trautwein, and Dave Chennault who fronted the event. Well over 100 companies attended, and feedback has been very positive. Everyone seemed to enjoy the virtual format, which is so convenient. Hopefully DynTek will invite us to the next one!

Tech.Ed Orlando 2008

I just spent the week in Orlando attending the sessions and helping to staff the echoTechnology booth. It was an excellent networking opportunity (as these things always are) and a great way to talk to people about their SharePoint implementations and pain points.

Garry Smith and RK Bommini

By now many organizations have had a chance to evaluate SharePoint 2007 and have either implemented a farm or are about to. One complaint that came up again and again from early adopters is that the portals "ran away" from them and had become a real mess to manage. Another is that it was becoming so hard to maintain the portal that administrators were starting to push back at the business requirements, with the result that the SharePoint implementation was not evolving as well as predicted.

In some ways SharePoint is a victim of its own success - it looks like a wizard-driven software suite and it appears that after a couple of clicks a portal is up and running and ready for the business to use. This is not true!

SharePoint is most interesting and worthwhile when it is helping an organization solve its enterprise problems - tackling big challenges like document management, information-worker collaboration, and enterprise search. These challenges are far more that just technical; addressing them requires a great deal of communication, focus, and effort on the part of the business. SharePoint isn't a silver bullet, or a shortcut to avoid those hard yards.

The word that kept coming up during the SharePoint sessions and on the show floor was "governance". It's a catch-all phrase that really means making the appropriate and ongoing investments of time, thought, skills, and money in addition to the upfront investment of SharePoint licenses and servers. Without that additional commitment, there is little point attempting a SharePoint implementation - it will be doomed to failure.

So what are some of the practical tasks that need to be undertaken as part of governance? Much more on this in the next few days...

I'm baaaaaack! Back from Oceania and freshly installed in my native city of Ottawa, Canada. The last couple of years have been a happy blur. I'll definitely miss Oz and the great people at Dimension Data Australia but it's nice to be in the same time zone as most of my friends and family.

As I mentioned in my previous post, I am working on some new projects.
One of these is collaborating with echoTechnology as their new Technical Product Manager to help develop and promote their tool, echo for SharePoint. This is like a Swiss-army knife utility suite that makes changing SharePoint easy. It's a product I've had my eye on for over a year, ever since I first interviewed the echo guys in these posts: echo demo and interview.

The reason I'm so keen on the tool, and why I agreed to work with them, is that after two years of SharePoint consulting I have yet to find a suite of tools that makes it so easy to manage SharePoint configurations and settings. Trying to migrate my solutions from Dev to Test to Prod has been an exercise in frustration. Keeping the state of a farm consistent when multiple team mates worked in parallel, pushing out changes to more than one site, or sharing the deltas with colleagues who are working in their own virtual machines has required a relentless search for scripts, code snippets, and any means possible to gain a little development consistency and control.

Part of my brief with echoTechnology is to work with the community to help develop best practices on SharePoint governance and change management. This is an area dear to my heart, and the focus of many of my workshops and engagements over the last few years. I've devoted a fair bit of space on this blog to those kinds of issues and that will continue.

Whenever possible I plan to release white papers, site templates, and snippets of code on this blog. I hope they will be useful to anyone planning to govern their SharePoint implementation - which should be EVERYONE!