SharePoint 2010: Search Service is not able to connect to the machine that hosts the administration component

Had a strange Search error in a new SharePoint 2010 farm:

The search service is not able to connect to the machine that hosts the administration component. Verify that the administration component 'bae8d161-c132-4570-aa0d-57baf6cf9d9b' in search application 'Search Service Application 1' is in a good state and try again."

I was unable to start any crawls as a result.

The search services were all started correctly but in the Application Services window it showed “Error”. The fix for me was to click on the Search Service Application name in the list of Application Services, choose Properties on the ribbon above, and change the Application Pool to another one (in my case, from SharePoint Web Services Default app pool to SharePoint Web Services System).

Hope it helps someone.

SharePoint Workspace 2010 “exceeds the lookup column threshold”

While trying to sync to a SharePoint site using SharePoint Workspace 2010, you may bump into the following strange error: "The query cannot be completed because the number of lookup columns it contains exceeds the lookup column threshold"

This is due to SharePoint’s new resource throttling settings for managing large lists. There is an easy fix in Central Administration.

Go to Central Administration and then browse to Application Management > Manage Web Application.  Select the web application you need.

In General Settings choose Resource Throttling. Set the value in the List View Lookup Threshold textbox to a higher value (equal to or greater than the number of site columns you are using on the list where you saw the error).

I bumped into this with Workspace but this error may appear for other client applications / systems that use SharePoint 2010 as a platform, and are trying to lookup lists that have more site columns than the throttle setting.

How To: In Place Record Views in SharePoint 2010

One of the great new features in SharePoint 2010 is the In Place Record declaration. This allows a user to apply Records management to any file in a document library.

To enable this feature, you must first activate the In Place Records feature in your site collection. Next, for a given library, go to its settings and choose “Record Declaration Settings”. Choose “Always allow the manual declaration of records”.

 

This will allow the Declare Record button to appear in the ribbon.

Now that you have a collection of records and documents in the library, you will probably wish to sort them using views. I like to create two extra views: Active (which means all active documents or “non-records” and All Records (which means anything declared as a record).

It is easy to create the views once you know the steps. The following steps come from my colleague Hoking:

To make the views, you need to first enable in-place records management in the library settings.  Then you need to actually declare 1 document as a record manually - this is in order to get the "Declared Record" field visible in the create/modify view pages.  For the Active view, filter on Declared Record equal to [blank], as in don't enter a value for the field.  For the All Records view, set the filter on Declared Record not equal to [blank].

Hope that helps!

MOSS 2007 Service Pack 2 Backup May Lock and Remove Site Actions Options

We had a very strange error suddenly appear on a client’s site last week – we got the following email:

Anyone know what's happening with site actions menu on the top level site of [the website]? I have limited menu options - i.e. there is no site actions, edit page, create page, create site, etc.

Our support team looked into this and figured out the following:

The issue appeared on the top-level site collection due it being set to “Read-Only” after a backup locked the site and then failed to release it. Changing it to “Not locked” immediately restored the menus.

This is what our guys had to say:

In SP2, Microsoft added a feature to STSADM to automatically lock the site collection during a backup:

http://technet.microsoft.com/en-us/library/cc287967(office.12).aspx

“In Service Pack 2 for SharePoint Products and Technologies, site collections are automatically locked as read-only before a backup process occurs. Therefore, there is no need to use the Setsitelock: Stsadm operation (Windows SharePoint Services) operation. If you do not want site collections to be locked as read-only, you must specify the nositelock parameter that is available in Windows SharePoint Services 3.0 with Service Pack 2 (SP2).”

It is likely that some kind of error or exception interrupted the scheduled backup, and as a result the lock was not removed.

Here is the page link: http://[your address]/_admin/sitequota.aspx.

Hope that helps others out!

SharePoint 2010 Bug: Cannot Rename Title On Validated List

My colleague David has noticed what seems to be a bug in SharePoint 2010 on lists that have validation applied to them. If you try to rename the title, it will cause an error. The "fix" is to remove the validation and then the renaming can occur as normal. We're doing some more testing to see if there is a workaround. It would be a shame as we are already using and enjoying the list validation functionality.

I'll update if we find a workaround.

Familiar Faces at the SharePoint Summit 2010

Just got back from attending the SharePoint Summit 2010 in Montreal with my business partner Keith Carter.

One of the nice things about the summit (apart from being in Montreal, which is always fun – awful traffic excepted) was that I saw a lot of familiar faces.

There were a lot of vendors I knew from my days on the SharePoint trade show floors – folks like Gail Shlansky, Yancy Lent, and Ken Allen from Axceler, Dave Seaman from Syntergy, Michael Potter from Nintex, and Tony Lanni from AvePoint.

Other familiar faces included speakers like Ruven Gotz and Wouter van Vugt.

This year, I had a happy surprise - a lot of my former colleagues from Parks Canada were present. We had worked together in the Office 12 TAP days when SharePoint 2007 was about to release and it was  a nice reunion.

Like any summit, it was also a chance to meet new folks. Before the summit kicked off, there was a dinner for all the speakers at the Greek restaurant Restaurant Ilios, organized by Danny Boulanger and Alain Lord. We had some nice chats about SharePoint, Holland, and Canadian culture (it turns out we do have one!).

There were a lot of Canadian government folks at the conference – it seemed like a majority of the participants in fact were public servants. There were some interesting chats about the challenges of buying and deploying SharePoint in the government.

On Wednesday I presented on migrating to SharePoint 2010.The interest in upgrading to SharePoint 2010 was high, so the room was full. I’m embedding my slide presentation below:

SharePoint Summit 2010 - Stress Free Migration

View more presentations from Nick Kellett.

It was a good conference and I look forward to the next one!

SharePoint 2010 Beta Migration Gotcha – SSP Errors

One of the big changes to SharePoint 2010 is the replacement of the Shared Service Provider concept with a series of application services.

Essentially instead of a central managed “pool” or bundle of application services such as Excel, User Profile, Search, and BDC, the new model allows farm administrators to turn on or off (and separately configure) an infinite variety of services that can add extra functionality to their farm.

This is a much more flexible model.

This new model does cause one problem if you are doing “in-place” migrations of existing 2007 portals to 2010 using the beta bits. For some reason, the Shared Service Provider database does get upgraded to 2010 but it causes a number of portal errors while it does so.

You’ll find these in your upgrade log file, in the 14 hive. These vary from User Profile errors to “certificate” errors. One common error looks like this:

[OWSTIMER] [UserProfileSharedResourceProvider12Sequence] [ERROR] [1/8/2010 9:23:42 AM]: Action 14.0.1.0 of Microsoft.Office.Server.Upgrade.UserProfileSharedResourceProvider12Sequence failed.

[OWSTIMER] [UserProfileSharedResourceProvider12Sequence] [ERROR] [1/8/2010 9:23:42 AM]: Inner Exception: There are no online service instances for this application.

The “fix” is to delete all Shared Service Providers from your 2007 farm BEFORE doing the in place upgrade. The sts command to do this is:

stsadm -o deletessp -title SSP -force

This isn’t a big deal - this is only a beta problem.I very much doubt this will be the case in RTM which is due next month.

Hope that helps.

SharePoint 2010 Migration – Stress Free!

SharePoint 2010 Migration Seminar

In my new role as Chief Technical Officer at StoneShare Inc., a Canadian SharePoint solutions firm, I am currently working on SharePoint 2010 migration options, and will be presenting a seminar on migration at several different events over the next month.

This topic will be focused on administrators and business users who want to understand migration paths and how best to prepare for a 2010 move.

I will do a live demo of an in-place SharePoint 2010 Upgrade - not for the faint of heart :) - and discuss the various options, useful tools, what sort of planning is required, and actual technical steps to get a great migration result.

I’ll be publishing my slides once the seminar is complete. Here’s an overview of the demonstration:

Migrate to SharePoint 2010 - Stress Free!

Considering migrating your current SharePoint environment to SharePoint 2010?  Worried about what’s involved and how to manage it?  Don’t let it become a headache!  This presentation will discuss some common sense business and technical approaches to take away the pain, and help you deliver your SharePoint 2010 migration project on time and on budget.

Topics Covered:

• A little history: The SharePoint 2003 to 2007 Migration experience
• Common Migration Pains
• SharePoint 2010 Technical Changes
• Governance
• Migration Options
• Migration tools and utilities
• The Migration Process
• Recommendations

I will be presenting at the following events:

SetFocus SharePoint 2010 Seminar

This is part of a four-part series of seminars on SharePoint 2010 that SetFocus is putting on.

When: Thursday, March 18 at 1 – 4 PM PST

Where: Online webinar. You can register for free here:http://www.setfocus.net/marketing/spseminarseries.aspx 

 

Microsoft Ottawa Federal SharePoint User’s Group

This is the monthly SharePoint user group – it is definitely open for more than just Federal SharePoint Users to attend!

When: Tuesday, March 30 at 5 – 7 PM PST

Where: Ottawa at the Microsoft Office on 100 Queen Street

SharePoint 2010 Summit

When: April 12 to 14

Where: Centre Mont-Royale, Montreal

Register online now for this great SharePoint conference in one of the world’s great cities.

http://www.sharepointsummit2010.com/index_e.htm

 

There’s really a lot to discuss and I’m looking forward to the seminars. I’m also crossing my fingers the live upgrade goes as planned each time :)

SharePoint 2010 Likely To Offer App Store

This just in from ReadWriteWeb:

Microsoft will offer an application marketplace within Sharepoint 2010 that will integrate with third-party applications from its partner network. No date has been set for the marketplace lauch but it will evolve from "The Gallery" a feature that provides Sharepoint 2010 users access to templates…

Details are few about the application marketplace that will be offered through Sharepoint. But it does point to the increasing significance of third-party applications for the Sharepoint platform and how the service may evolve as cloud computing becomes more prevalent.

I was predicting this a few weeks ago on my “Things To Get Excited About in SharePoint 2010” post. Here’s what I had to say:

Service Application Architecture – the Shared Service Provider was a good idea but it was a bit hard to use in practice. Under the new architecture, you can create Service Applications for things like Excel Services, Forms Services, Business Connectivity Services, and other services that you build or buy, and you can mix and match these in your farms as you like. The services get consumed by web front ends via a standard interface.

This should allow a lot of plug-and-play customization of farms. I’m even wondering if there is an opportunity for vendors here…create some services and expose them to clients from the cloud.

There are some other big changes like Claims Based Authentication and Solution sandboxing which are intriguing to me. The Solution sandboxing feature gives me this sneaking suspicion we will one day soon see a Microsoft SharePoint App Store where we can buy, download and run SharePoint solutions in our farms.

Magic Eight-ball now says: “You may rely on it”.

Hosting Clockwork Web Framework With Amazon

I’ve blogged a lot about my admiration for Amazon’s web services stack. I think they understand the web as well as any company in the world. It’s always been my intention to investigate Amazon’s Electronic Compute Cloud (EC2) and since I needed hosting for my new Clockwork Web Framework, I decided to give it a try.

The reason I went with Amazon rather than a traditional hoster is that I have no idea what kind of interest there will be in the framework, and therefore cannot predict what the load on a web server will be. Amazon EC2 is designed for this kind of flexibility, and you pay per hour.

The Platform

I am running a small Windows Server 2003 32x server instance to begin with. It only has 1.7 gb of RAM. I can scale this up if I need to, or more likely I will run up another small instance and load balancing the two using Amazon’s Elastic Load Balancer technology.

On this, I am using IIS 6, .NET 3.5, SQL Server 2005 Express, and Powershell. Most of my files are kept on a permanent storage drive (more on this below) and served by IIS. In order to maximize the speed and lower the CPU burden on the server, I have decided to use another Amazon technology, CloudFront.

CloudFront Content Delivery Network

CloudFront is a Content Delivery Network (CDN), like Akamai or Limelight. I use it to serve my images and resource files. Basically Amazon has edge servers all over the world with a copy of my images and resource files, and when users request them from my website, CloudFront automatically sends them a copy from the nearest location to them, making for some very fast download times.

To make this work, you have to use Amazon Simple Storage System, or S3. This is a virtual file system. Basically you have “buckets” of files that are served up when requests come in from the CloudFront “distributions”.

I’ve optimized it a bit by having two distributions; one for images and one for resources. This means that a page which requires both things will load even faster since two parallel CDN distributions are processing the files at the same time.

You can create CloudFront distributions through code, or through Amazon’s web management portal.

  Since you can control the public URL of the distributions, you will notice if you view the properties of my website that my images are handled by the path “http://images.clockworkwf.com” and my resource files are handled by the path “http://resources.clockworkwf.com” . In other words, I have full control over what path I give them. Most people will never know these picture are being served from Amazon.

I notice the website loads really quickly, so the CloudFront makes a big difference.

EC2 Hosting Challenges

So that’s the high level architecture. There are a number of impacts when using Amazon as a hoster I’d like to talk about.

Server Goes Up, Server Goes Down

To begin with, you have to assume that at any moment your server will go down. If your server dies, it vanishes, and you have to “spin up” another one, using the web interface or code. It’s very easy to do from the web console, just click “Launch Instance” and you can pick any server ranging from Ubuntu Linux to Windows 2003 Server 64x Enterprise R2.

Although the server instances you can use have their own hard drive space on C: and D: drives, you have to treat that as transitory storage.

I’ve setup my system in such a way that I can use an Elastic Block Storage (EBS) hard drive volume, provided by Amazon.This is a more permanent drive space that you pay for, but can be attached to any server instance. Think of it as a SAN (that’s probably what it is).

So I’ve got my database and web files on this EBS block, which I then mount to any server instance I’m currently running.

On the server instance, I simply point IIS web server to the EBS block files, and away we go.

The EBS can be any size you like, and you pay per GB per month. Right now I’m using 10GB since my log files and database don’t take up much room. I can add more space later if I need to.

Here’s a screenshot of that EBS volume, in the Amazon web console.

Dynamic DNS Entries

Next problem: Since the server can go down at any moment, DNS is a problem. If my server dies and I spin another one up, it will be given its own IP address, which my DNS entry for www.clockworkwf.com wouldn’t know about. So there might be a long delay while DNS changes to the new IP address.

So, I’m using a Dynamic DNS service called Nettica. They have a management console where I can enter my various domain records and assign a short Time To Live (TTL), which means the DNS entries update frequently. So if my server dies, I can change the entry in Nettica to point to the new server’s IP address, and within a few seconds requests are going back to the right place.

Nettica even allows me to control all of this through C# code. Going forward I plan to write powershell server management scripts that can automatically spin up a new server on Amazon, determine the IP, and register that with Nettica.

Incidentally, Amazon EC2 allows you to buy what are called “Static IP Addresses”. Essentially you can “rent” a fixed IP address which can by dynamically allocated to a server instance. So, in the short run this makes life easier for me as I have rented one, used that for my Nettica domain name record, and can assign this fixed IP to any new server instance.

Next problem: Disaster Recovery.

Disaster Recovery is even more important in Amazon EC2 world than elsewhere, since again your instances could die at any moment….Not that they will, but the point is, they are “virtual” and Amazon isn’t making any promises (unless you buy a Service Level Agreement from them).

However, Amazon’s EC2 provides a level of DR by its very nature – you can spin up another machine in a small amount of time. Estimates for new Windows instances are about 20 minutes.

There’s also something called an Availability Zone. Essentially it means “Data Centre” – Amazon has several of these and so you can spread your servers around between US – East, US-West, Europe, and so on. So when that Dinosaur-killing comet hits North America, the Europe Availability Zone keeps chugging.

Right now I’m not really doing much with my database, so DR isn’t such an issue. I have some security since my files are on an EBS block. However, eventually I’ll setup a second server in another availability zone and load balance the two.

Another Challenge: Price

Amazon Web Services are flexible, and you are charged per hour, for only what you use. This is an amazing model but it doesn’t work so well for website hosting, because of course your servers are supposed to be online 24/7, 365 days a year.

It’s hard to tell for sure what the annual bill will be, but for my small server instance (remember, only 1.7 Gb of RAM) it will cost well over $1,000. That’s a lot more than shared space on a regular hosting provider. However I’m willing to pay this, for the flexibility I get, and also because I think Amazon web services are a strategic advantage and so the earlier I learn about them, the more business opportunities I might unlock.

One good thing is that Amazon has been aggressively dropping its prices as it improves its services. Additionally, they have started offering “Registered Servers” – basically a pre-pay option for 1-year, 2-year, and 3-year terms. Unfortunately these are only for Linux servers at the moment but hopefully they will add them for Windows and then I can save money year on year.

CloudHost Monitoring

Amazon offers a web-based monitoring option for its server instances. I’ve started using it (for an additional fee) but I’m not sold on its utility yet. I don’t think I’m using it to its full potential yet – it is supposed to help you manage server issues by monitoring thresholds.

Managing S3 Files Using Cloudberry Explorer

I needed an easy way to create and manage my buckets, CloudFront distributions, and S3 files. I found Cloudberry Explorer, and downloaded the free version of it. I was able to drag and drop 1600 files from my Software Development Kit to the S3 bucket where I’m serving the resources. Super!

There’s a pro version I might purchase which would allow me to set the gzip encryption and other properties on the files. This would help lower my bandwidth costs and speed up the transfer a bit.

Here’s a screenshot of Cloudberry in action:

I love how easy it is to setup and use Amazon’s web services stack. I think they have a great business model for the Cloud, and they’re the company to beat. I’m willing to rely on them for the launch of Clockwork Web framework and so far I haven’t been disappointed.

Introducing Clockwork Web Framework for .NET

In 2003, I read a book, “Making Space Happen”, by Paula Berinstein. It’s about the efforts of entrepreneurs to open up space to the public. It’s the kind of thing that gets my propeller-head spinning, and after reading it I resolved to create the best website on space travel on the internet.

So, I sat down in a park and within two hours I had covered several sheets of paper with scribbles and scrawls of what my website needed. I had notes on authentication, web components, search boxes, themes, dynamic images, language toggles, and all kinds of stuff.

Being a good little programmer, the more I designed, the more intricate the design became, and pretty soon I was knee-deep in code. Flash forward six years later, and I have yet to write a single page of that space website!

But I do have a web framework :)

What It Is

Clockwork makes it easy to build powerful .NET web sites. It’s completely free, open source (under the Apache 2 license) and you can use it in proprietary or open source projects, as you like.

Some of the ways it makes web development easy:

• Database-agnostic data access
• Dynamically displays content in different languages
• Leverages the .NET 3.5 framework, including the Provider Model, generics, LINQ, automatic properties, and more
• Integrates with popular web services such as those provided by UserVoice, LinkedIn, Google and Yahoo!
• Makes it really easy to use object-oriented programming standards like Dependency Injection / Inversion of Control, Repositories, and Specifications

Under the hood I use many popular components, including NHibernate for database access, Castle Windsor for Dependency Injection, and log4Net for logging.

Although today marks the official public release, the framework is currently at version 3.x because I’ve been using earlier versions of it in production websites since 2004.

I’ve built Clockwork using as many web standards as I can find, as many of the latest .NET elements as possible, software best practices, and a lot of love and stubbornness.

What It Will Become

Well, it’s obviously too early to say. But I am committed to continuing to develop it, I have a long list of things I plan to add, and I’m hopeful a community of .NET developers will adopt it and push it into areas I can’t even imagine today.

Please take a minute to visit the website and learn more about it. I hope you find it helpful.

Many thanks,

Nick

Central Administration in SharePoint 2010

Here’s a quick lap around the new Central Administration console in SharePoint 2010.

New Central Administration Layout

The navigation structure is broken down a little more than in 2007. There is no more “Operation” and “Application Management” divide; instead the new console is divided into the following sections:

• Application Management: Manage site collections, web applications, content databases, and the new service applications
• System Settings: Manage servers, features, solutions, and farm-wide settings
• Monitoring: Track, view and report the health and status of your SharePoint farms
• Backup and Restore: Performs backup or restores
• Security: Manage settings for users, policy, and global security
• Upgrade and Migration: Upgrade SharePoint, add licenses, enable Enterprise Features
• General Application Settings: Anything that doesn’t fit into one of the other sections
• Configuration Wizards: These are nice wizards to help setup or modify the farm

This is new layout is an advantage – the “Operations” and “Application Management” tabs in 2007 always felt a bit arbitrary and it wasn’t always clear which tasks went where.

Monitoring

This is quite useful – basically you can take the heartbeat of SharePoint and its services via reports, and view problems and solutions. Here’s a screenshot of the interface:

There are only a couple of reports right now, which tell you which pages loaded the slowest, and which users are the most active. I imagine for release there will be many more.

The problem and solution report is very helpful in identifying which services are failing on which servers, and why. Notice in in this report there is detailed information about one of the failing services, in this case Visio, and links to remedy it.

Surfacing common errors in this way will go a long way to reducing the IT administrative burden of SharePoint. I hope Microsoft is active in populating this report engine (or provides a way for the community to modify it).

Usage logging settings are in here as well.

Service Applications

These new plug-and-play replacements for the Shared Service Provider are major wins for the new SharePoint version. They allow an organization to really customize its farm based on its needs and even usage patterns. Services that needs lots of performance and support can get it, while services that are less useful can have reduced resources or even be turned off altogether. Everybody’s SharePoint 2007 farm looked alike, but going forward it is likely that no two farms will be alike.

Of course to manage this Microsoft has to surface the available services and their settings in the Central Admin. This screenshot gives an indication of just how many services can be used.

Export Sites and Lists

Now you can export site and list data right from SharePoint! It’s straightforward with the new Backup and Restore section, which allows full Farm Backups and Restores along with far more granular backup. The backup can include full security including site users, as well as version history information for each item in the list.

I doubt this will replace the need for 3rd party backup software but it’s another tool for IT Admins.

Here I am backing up a Calendar from a site to file.

The new service architecture of SharePoint is one of the most exciting things about it, and obviously required a bit of a Central Administration retooling. That provided an opportunity for some other quick wins, including a much more intuitive navigation structure and some neat monitoring tasks. More evidence that SharePoint 2010 is building on, but not replacing, the core strengths of 2007.

When SharePoint 2010 Met Web 2.0

One of the goals in SharePoint 2010 was to make it easier for users to update their information and pages without lots of postbacks, clicking, and delays. Accordingly, Microsoft has invested a lot in improving the web user interface.

One way they have done this is by adding the Office Ribbon concept to SharePoint. I think this has to be a first for a web application, and to be honest while I saw the value in Office 2007, I wasn’t sold on it for a web interface.

I think the major weakness of the Ribbon concept is that you can spend a fair amount of time trying to remember what command belongs to what tab. As well, it doesn’t always save clicks. More on that in a moment.

The other major investment Microsoft made is adding AJAX. This is  no-brainer and a hands-down winner for me. I’ve attached some screenshots to show how you would modify a page in the new UI.

Let’s imagine you want to modify a team site:

Step 1: You are in the Browse tab of the Ribbon (up top) – choose the Edit Tab.

To Edit, click “Edit” which is one of the buttons on the Edit tab. Then click on the area of the page you want, type some text in, and click Stop Editing. Are we saving clicks yet? :)

Well, not so far, but there weren’t any postbacks, so overall I think there’s some time saving here. An important benefit from a training perspective is the server and office products now have identical user experiences, which is a big win.

As well, there are some nice new options including an XHTML converter. And did I mention this all works flawlessly in FireFox? Web standards, hooray!

You can also insert new web parts via the Insert section of the Edit Ribbon:

Of course, the context-based Ribbon experience continues when managing lists and libraries. Here’s a screenshot of the out of the box Shared Documents library’s two important ribbons, Documents and Library:

Finally, tagging and sharing is a major concept in Web 2.0 and SharePoint 2010 addresses this by surfacing sharing activities through the Ribbon. Content can be easily tagged - Tags can be private or public and are automatically added to a suggested set so that users can share tags.

Tagging is also part of a user’s Activity Stream (not sure what the official term is). You can see on my profile that I tagged an element.

I’m not showing it here but there is also an Enterprise Metadata service that allows an organization to centrally control its taxonomy. So, now you can make peace between the “folksonomy” and “centralized taxonomy” gangs in your office!

All in all these UI improvements are icing on the SharePoint 2007 cake. I’m not sure they are enough by themselves to encourage SharePoint 2007 customers to upgrade (I think there are better reasons to upgrade), but somebody with 2003 or without SharePoint at all might now make the plunge. However, these are welcome additions to an already great product.

Although I’m not convinced the ribbon will save clicks, and will certainly take some retraining and familiarization time, it at least is consistent with the Office clients, making for tighter integration. The AJAX-style UI is a big win, and the inclusion of some interesting tagging and sharing features brings SharePoint up-to-date with the Web 2.0 world.

Things To Get Excited About In SharePoint 2010

Now that Microsoft’s lifted the TAP NDA and is presenting SharePoint 2010 publicly at the SharePoint Conference in Las Vegas, there will be a spurt of queued up blog posts on the net :)

Here are some things I’ve been very excited about, in no particular order. They are fairly developer-centric.

• Ability to develop against the SharePoint dlls on a developer desktop! ‘Nuff said.
• Developer Dashboard – makes it easy to see tracing information and web server details when you are working on a SharePoint site.
• LINQ to SharePoint – this is some nice syntactic sugar that helps replace CAML a little bit. You can created strongly typed SharePoint entities using a utility called SPMetal and then query and manipulate the data in them using standard LINQ syntax. I was hopefully predicting this in another post.
• Visual Studio 2010 integration – VS2010 will have a lot more tools to make SP2010 development a snap. SharePoint Project and Item Templates, Feature Designer, and Project Packaging, will hide most of the messy details of creating, packaging, and deploying a SharePoint solution from the developers.
• Business Connectivity Services – the next level of the Business Data Catalogue. BCS uses External Content Types which look a bit like Content Types, and are defined in the new SharePoint Designer or in Visual Studio and then added to SharePoint using a definition file (a bit like the BDC currently works). Users can then create External Lists in their sites, which pull in the data from these external sources.
• Client Object Model – an abstraction layer that allows developers to write code that will work in client .NET applications, Javascript (for AJAX type operations), and Silverlight. Basically this is a disconnected, batch-style API that will operate on the existing SharePoint web services and handle requests and responses using XML and JSON.
• SharePoint 2010 Designer – Whereas SPD 2007 was a warmed-over FrontPage, the new version has been rebuilt with a focus purely on SharePoint. The new navigation panel is great because it shows you a list of SharePoint objects, such as Entities, Lists, Master Pages, and Workflows. What’s great about this is it keeps you thinking about what you are trying to do in SharePoint, rather than where that command used hidden in SPD. Another big win is you can export your SPD changes as a .WSP file straight into Visual Studio for further customization.
• The Office Ribbon makes it into SharePoint. The Ribbon kind of grew on me in Office 2007. I think it was a clever paradigm to surface many commands that used to be buried. Now the many SharePoint menus and Site Action dropdowns will coalesce into the Ribbon. I think this will make training and support a little easier. The big weakness of the Ribbon is that you often have to remember which tab the commands belong in. I found that was the case with the new SharePoint Ribbon but after a little while you get used to it, and it becomes faster to modify SharePoint pages.
• STSADM is dead, long live PowerShell! Leveraging the great new scripting environment is a huge win for SharePoint. The ability to write .NET code to manipulate the command pipeline means we will start to see some very powerful “no-touch” deployment and management options for SharePoint
• More events – now you can find out when your web or list was created or deleted. This may sound like a small feature but this enables some provisioning and discovery scenarios that in SP2007 were not even possible!
• Enterprise Metadata Manager. I’ve blogged a lot about the important of governance and centralizing metadata. The new Enterprise Metadata Manager makes it easy to import and manage term sets, keyword and tags.
• Service Application Architecture – the Shared Service Provider was a good idea but it was a bit hard to use in practice. Under the new architecture, you can create Service Applications for things like Excel Services, Forms Services, Business Connectivity Services, and other services that you build or buy, and you can mix and match these in your farms as you like. The services get consumed by web front ends via a standard interface. This should allow a lot of plug-and-play customization of farms. I’m even wondering if there is an opportunity for vendors here…create some services and expose them to clients from the cloud.

There are some other big changes like Claims Based Authentication and Solution sandboxing which are intriguing to me. The Solution sandboxing feature gives me this sneaking suspicion we will one day soon see a Microsoft SharePoint App Store where we can buy, download and run SharePoint solutions in our farms.

Anyway, there’s a lot of exciting new stuff in SharePoint and I think SharePoint development is about to become really fun!

SharePoint: A Product and a Platform

SetFocus just published another of my articles for their Technical Articles section. This one is called “SharePoint: A Product and a Platform”, and discusses the implications of SharePoint as a software platform.

My conclusions are that the platform provides significant capabilities including a unified development environment, reduced maintenance, development, support, and training costs, and may increase the risk of vendor lock-in.

I’ve written for SetFocus before because I have a long association with them, dating back a decade. I had my Java certification training and first job placement through them. For the past year I’ve been developing and teaching parts of their SharePoint programming classes for the SharePoint Master’s Program (I’m instructing evening classes again starting this Saturday).

You can read more at http://www.setfocus.com/TechnicalArticles/Articles/sharepointproductandplatform.aspx. I hope you enjoy it and welcome your feedback!

P.S. The article is licensed under the Creative Commons Attribution-Share Alike 3.0 Unported License which means you can modify it and share it around!

NHibernate Performance Profiling with NHProf

NHibernate

I’ve been using NHibernate a lot recently. It’s an Object Relational Mapping software that makes it easy to “map” between SQL database syntax and standard C# object models. The goal is to talk to databases, by writing code like this:

var query = session.CreateQuery("from WebPage p where p.VirtualPath like :path")
.SetString("path", "%pages%")
IList<WebPage> list = query.List<WebPage>();

Now behind the scenes, there’s a relational database somewhere – and transactions, and validation, and syntax parsing, and query analyzing, and all of that standard relational database stuff – but as a programmer I just need to know about my object model and it will return me a list of WebPage objects and I can easily use them in my code, update them, and delete them. Shiny!

NHibernate is a straight port of Hibernate, from the Java world where it originally evolved many years ago. So the concepts behind it have been field-tested for in both Java and (now) .NET shops. This makes it a very robust ORM tool. Did I mention it is completely FREE?

While it’s amazing software, it comes with a big learning curve. There isn’t much documentation out there – and most of it is on blogs and wikis. I’ve bought Manning’s NHibernate In Action and that helps a bit. However, there isn’t much information on common performance and configuration traps.

Learning and Analyzing With NHProf

So I was glad to find out about NHProf, a profiling and analyzing tool for NHibernate created by one of NHibernate’s main developers (Oren Eini aka Ayende Rahien). His colleagues are Christopher Bennage and Rob Eisenberg.

Essentially the profiler is a slick-looking Windows Presentation  Foundation executable that “records” your application as it writes statistical data to the NHibernate log file, then provides a graphical view of the various things that are going on under the hood.

The interface is well thought out, with only a few tabs and windows, so the information is easy to sort through. Here’s a screenshot of the main interface:

What I Like About It

Now to the things I really like about this software:

First, you can see the exact SQL query that NHibernate is generating. Straightforward, but critical. There is a related Stack Trace which allows you to jump to the part of your code where you executed this statement.

As well, you can view the rows that are returned by any query. This makes it easy to see exactly what data you are getting back – a much-needed sanity check at times :)

 

Each NHibernate action is evaluated against known best practices (or bad practices) and you get “Alerts” that can provide more information on what to do (or not do).

For example, while running some recent queries, I received the following alert: “Too many cache calls per session”.

 

This leads me to the final element that I LOVE – the “read more” and “NHibernate Guidance” features. Software is so complicated that I just want to get it working most of the time – but I know that if I really understood it, I would avoid a lot of bugs and future issues.

So what makes this software shine for me is the care that has gone into helping people learn NHibernate. By clicking “read more”  you go straight to a web page that teaches you about that particular error and ways to avoid it – including code samples!

 

As well, there is a “Guidance” option that you can always access to learn about general NHibernate performance issues such as “Select N+1” or “Unbounded Result Set”. I’ve already applied the lessons from “Unbounded Result Set” and “Do Not Use Implicit Transactions” to my code and the result is much better performance and stability.

One thing I would like is the ability to hover over an alert in the statement in the main window, and actually see a tooltip of the alert message. At the moment you can see the icon showing the alert, but then have to click on the statement and then click on the “Alerts” tab at the bottom to see what it’s for.

NHProf is still in final beta but I have been using it for about a month and have found it to be very stable. I just bought my copy – there is a discount right now before it hits RTM and I think it has already been worth the money.

I would recommend this to anybody using NHibernate.

Data Splunking

I’ve had my head down for the last couple of months, churning out code for that elusive framework I keep hinting at :) Right now I’m staying in a trailer with no tv, internet, or cell phone coverage and I’ve never been more productive (says I).

Still, I thought I would pop up briefly to mention a cool IT tool that can provide you with a centralized, browser-based repository to search on all the millions of log files, event viewers, and databases that are inevitably scattered around any company’s data centres.

It’s called splunk. Its name is clever – users get to spelunk into their data silos and see what’s there. It’s a simple, single package install that runs on most desktop machines and servers. There’s a free version if you use less than 500 megs of indexed data, and enterprises can pay to index larger corpuses. I’m running that on my Vista 64 bit box and it indexes and searches like a little champ.

In my case I’ve been using it on my framework log files to help analyze bugs and performance bottlenecks. Here’s a screenshot of a search on the keyword “nhibernate” (NHibernate is an Object Relational Mapping software):

As you can see, it quickly pops up all the logged events where NHibernate was called from my classes.

To get this to work, all I had to do was add an “Input” for splunk to index – in this case the full path to my log file folder.

As you would expect, it does lots of reporting. It has broken down my log files into various columns. Examples of these columns are: custom C# properties I search on; the standard log file “stuff” such as the source name, date created, file size; even the sql commands that NHibernate generates for me. I can filter these columns for even more detailed breakdowns. In the next screenshot I am reporting on Entity ID values I use to track my framework objects.

I like splunk because it’s a one-stop shop for me to analyze all my various bits of IT Operations information. There’s a slick AJAX web user interface, and so far performance seems fine for me on my little dev laptop. I find it solid, intuitive, and I don’t have to expend much effort to install, manage, or learn it.

There’s also a way to extend splunk using its custom Application Programming Interface. I plan to investigate that when I have some free time but have not had a look yet.

I think any IT company should give splunk a test run.

WCF WTF

Today I was working on a WCF application integration layer for my framework. For some bizarre reason, no matter how many times I updated the service contract and rebuilt my client service references, I never got the latest version.

Specifically, I was trying to call the following method:

public void CreatePages(IList<WebPageDTO> pagesToCreate)

The WebPageDTO was declared (temporarily) in the WCF project. I put DataContract and DataMember on it, the project built fine, and I could even add a reference to it from a client application. However, I could never update the reference from my clients, and when trying to view the service directly I would get this error:

The type '[namespace].[servicename]', provided as the Service attribute value in the ServiceHost directive could not be found.

Finally, I had to modify the method signature to RETURN WebPageDTO – which I didn’t want to do. However, that fixed the service reference and now I can update it. Now I have switched the method stub back to return a custom ServiceResponse object, and that also works.

Bit of a head scratcher. I don’t fully understand why this is happening. Anyone know?

Trawling for Business Offshore

On the High Seas, large ships drag vast nets along the ocean bottom, hoping to catch schools of helpless Fish Sticks. The IT equivalent are Offshoring Companies who cold-call businesses, trying to dredge up remote contract work.

Times must be tough for them - not a week has gone by without one of these firms ringing me out of the blue.

They always seem to follow the same screwy script. Somebody with a ludicrous title such as “Business Development Manager for Premier International Clients” introduces themselves as “Steve”, “Mike” or “Jane”. They’ve usually got thick Indian or Eastern European accents – perhaps their names really are Mike Jane or Steve but I’m guessing they think I’m somehow more likely to talk to them if I think they look exactly like me? That’s slightly insulting ;)

The conversation normally goes like this:

“Steve”: Hi, is this Nicholas?

Nick: Yes, speaking.

“Steve”: Hi, I am Steve from [Name withheld to protect the guilty]. I had a detailed look at your website and your focused approach towards Microsoft Technology. This focus presents tremendous opportunity between our companies to establish a mutual association towards IT outsourcing. [Name Withheld] is a mature, 200 person strong organization based out of [Somewhere]. We provide deep software engineering skills to the global software industry. We have a satisfied client base of more than 70+ companies spread across US and Europe.Our service areas include bespoke software development and testing services. We have a proficient team of IT professionals working in .Net (C#, ASP.Net, VB.Net, VB)  and SharePoint technology. We are working on few of the latest technologies like AJAX, JSF and Web Services.

Nick: Err, we’re not planning…

“Steve”: We also have expertise in XML, SilverLight, IIS 5, 6, and 7, Windows Start button, SQL Server Database, Oracle Database, Access Database, MySql Database, Flat Files…

Nick: Please hold on, we’re not looking to outsource…

“Steve”: …in addition to client software experience customizing Word, Excel, PowerPoint, Clippy, Microsoft Bob…

(10 minutes later)

“Steve”: …As well as Binary Load Lifters, baking, and cattle rustling.

Nick: Umm, I was trying to say that we are not planning on outsourcing our cattle rustling. Thank you and goodbye.

“Steve”: Thank you - I look forward to speaking to you again very shortly, Jonathan.

Nick: (Dial tone)

…

Seriously, is this how we’re supposed to do business? Two companies that obviously don’t know the first thing about each other are expected to work across time zones and culture barriers to create complicated software on time and under budget? Riiiiiiiiiiiight.

I’m not against outsourcing firms, but can you please approach me in a professional manner that indicates you are actually interested in working with me?

Actually the first outsourcing firm that called me has a reputation as scam artists. I even found a website run by outsourcing employees who named it as the worst company to work for! This company wouldn’t take no for an answer and months later were still calling me up and trying to get me to start a project with them (asking for 25% of the money up front, which was apparently their scam). So these unsolicited calls can be funny but annoying and scary at the same time.

Still, the fact they’re doing this means that – like Viagra spammers – they must somehow be getting some bites. So if you can’t beat ‘em, join ‘em. I’m going to start my own cold-calling to drive business partnerships:

“Nick”: Hi, is this Jeff Bezos?

Jeff Bezos: Speaking.

“Nick”: Hi, I had a detailed look at your website Amazon.com while I was buying DVDs on it, and really liked your focused approach towards e-Business. This focus presents tremendous opportunity between our companies to establish a mutual association towards IT outsourcing. We have a proficient team of IT professionals working in .Net (C#, ASP.Net, VB.Net, VB)  and SharePoint technology…

Jeff Bezos: (Dial tone)

“Nick”: Thank you for your time, Bill.

…

“Nick”: Hello, Larry Ellison? I had a detailed look at your website MySql.com …

Larry Ellison: (Dial Tone)

…

So, anybody else getting caught in the net?