How to query cross-site lists in DataFormWebPart - Part 3. Filtering on column headers problem

Here are the links to previous two parts of the series

Part 1. Build your own data source for Data Form Web Part

Part 2. Use XSLT generated from SharePoint Designer to display data 

If you build a Data Form Web Part with customized data source (i.e. data source that does cross-sites querying), you've probably noticed the filtering on column headers does not work. In this part of customize Data Form Web Part series, you will see how this problem can be fixed by extend your custom Data Form Web Part code a bit more.

Before click on "Form Name" column header, there are two items displayed under it.

James Tsai .Net Blog - SharePoint C# ASP.NET VSTO -  Data Form Web Part with cross-sites query data source displayed

After click on "Form Name" column header, no available filter data returned.

James Tsai .Net Blog - SharePoint C# ASP.NET VSTO -  Cross-sites query data source with filtering on column headers problem

The Problem

Before we going into how to fix it. You must first understand what is causing this problem. And the answer simple. Because with the customized data source you just created, the original Data From Web Part methods that used to handle filtering are no longer to be able to perform filtering and render filter value correctly.

You will see how it works behind the scene from the following three methods. These methods are called after you clicked on any filterable column header.

protected virtual void RaiseCallBackEvent(string eventArgument){};
protected virtual string GetCallbackResult(){};
private string DoCallBackFilters(string filterStr){};



This method assigns event argument to the web part (Data Form Web Part in this example) after callback event has been raised.

Event argument is a string variable represent in following format: (i.e. clicked on "Title" column header)

"__filter={Form Name @FormName x:string;1033 g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb}"

As you can see, it contains column name and web part ClientID - g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb.



After RaiseCallBackEvent, this method gets called. It first check if raised callback event is for filtering event on column headers. If so, it strips out the filtering string

"Form Name @FormName x:string;1033 g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb"

And pass it to DoCallBackFilters as an input argument.



DoCallBackFIlters prepares callback result that will be used for rendering filtering drop down list on column header. The correct returned string should look like this:

<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=##dvt_all##}')"></OPTION>
<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=Temp page}')">Temp page</OPTION>
<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=Test Page}')">Test Page</OPTION>

Note! This is where you get empty result when you clicked on column header. Because this method cannot process your customized cross-sites query data source, it returned empty string instead to GetCallbackResult.
James Tsai .Net Blog - SharePoint C# ASP.NET VSTO -  GetCallbackResult method failed on return correct data



DoCallBackFilters is a private method and there is no way to extend it to make it work the way you wanted. The only option here is to override GetCallbackResult method and make it to call new DoCallbackFilters replacement method. (CustomizedDoCallBackFIlters in this example)

/*Original implementation of GetCallbackResult*/

public virtual string GetCallbackResult()
    if (string.Compare(this.eventArgument, 0, "__filter", 0, 8, false, CultureInfo.InvariantCulture) != 0)
        return string.Empty;
    string filterStr = this.eventArgument.Substring(10, this.eventArgument.Length - 11);
    return this.DoCallBackFilters(filterStr);

Override above method with following implementation.

/*Override implementation of GetCallbackResult*/
public virtual string GetCallbackResult()
    //This is where it make sure event raised from filtering on column headers
    if (string.Compare(this.eventArgument, 0, "__filter", 0, 8, false, CultureInfo.InvariantCulture) != 0)
        return string.Empty;
    string filterStr = this.eventArgument.Substring(10, this.eventArgument.Length - 11);
    return this.CustomizedDoCallBackFilters(filterStr);
private string CustomizedDoCallBackFilters(string filterStr)
    string yourDataAsString;
    //Build your customized filtering data here. To return data in format described above
    return yourDataAsString;

Inside CustomizedDoCallBackFilters method you can build result data with any logic you like. As long as it is in correct format. I suggest you to follow the same logic of how you've built your data source. To make sure data displayed in column headers always consistent with actual displayed items in Data From Web Part.

James Tsai .Net Blog - SharePoint C# ASP.NET VSTO -  Correct data returned from GetCallbackResult method 

More Details

I will explain what should your final result data look like. As you can see in above, it follows this format:


<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=##dvt_all##}')"></OPTION>

<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=Temp page}')">Temp page</OPTION>

<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=Test Page}')">Test Page</OPTION>


Each <OPTION/> element inside <SELECT/> has only one attribute "herf". Except first <OPTION/> element served special purpose and I will explain it later.

But first let's see the break down to the "herf" attribute value

James Tsai .Net Blog - SharePoint C# ASP.NET VSTO - Correct result format explained

Only difference in each line are the values used to display on the UI drop down list and used for filtering PostBack result.

The first <OPTION/> is used to indicate whether filtering has been done on current column. To toggle the flag you just need to assign 1 to this <OPTION/> element

<OPTION href="__doPostBack('g_548009f0_beeb_4e3c_a4a6_71fc338cc8cb','NotUTF8;__filter={@FormName=##dvt_all##}')">1</OPTION>

James Tsai .Net Blog - SharePoint C# ASP.NET VSTO - How to use first element in returned result to toggle filtering on column header flag



That's all. It is pretty much how you fix filtering on column header problem. Just remember to return your final result data in GetCallbackResult().

In summary.

1. Get Callback argument with all the necessary data (field name and control client id).

2. Use above input argument to build result data using any code logic you like.

3. return data in GetCallbackResult() method.

Hope this post helps you to get what you want.




In Memory of - Lee Marriage 1982 - 2008

Lee Marriage - a colleague of mine passed away yesterday while competing in City to Surf. (News here).

He was SharePoint Technical Lead for the project I am currently working on. A friend, true leader and SharePoint Guru.

Sure it is loss to SharePoint Community, especially he was the one who has helped to organize Sydney SharePoint User Group each month.

I know I am going to miss your UK accent, your joke and your smiley face everyday when I walk in office to work.

Thank you for buying me coffee last week after both you and I returned from leaves to work. I've learnt a lot from you (Drinking is one of them)


(CodePlex) MOSS Faceted Search - HTML encode/decode problem fixed!

MOSS Faceted Search is a great SharePoint search extension project on CodePlex. It provides following features (copy from its CodePlex site)

  • Grouping search results by facet
  • Displaying a total number of hits per facet value
  • Refining search results by facet value
  • Update of the facet menu based on refined search criteria
  • Displaying of the search criteria in a Bread Crumbs
  • Ability to exclude the chosen facet from the search criteria
  • Flexibility of the Faceted search configuration and its consistency with MOSS administration

However, there are still some known issues exist due to this project does not update and release regularly.

HTML Encoding

An issue I experienced was HTML string encoding/decoding problem. After I added faceted search web part to search result page and searched for something, it returns result like following.


James Tsai .Net VSTO SharePoint Blog MOSS Faceted Search problem 


As you can see first facet value in both "Refine by Title" and "Refine by role" sections are empty. Only number of hits displayed. And it is because Faceted web part does not perform HTML encode/decode properly. It can be easily spotted by closer inspection of page's HTML source.

<!-- rendered html of facet web part -->

<td nowrap="nowrap" width="100%" style="vertical-align:middle"><a class="ms-navitem" href="javascript:__doPostBack(...)" style="vertical-align:middle;"><No Title></a><span> (2)</span></td>

<td nowrap="nowrap" width="100%" style="vertical-align:middle"><a title="&amp;lt;enter position title>" class="ms-navitem" href="javascript:__doPostBack(...)",style="vertical-align:middle;"><No Position defi...&lt;/a><span> (1)</span></td>

First empty facet value is causing by "<No Title>" rendered as html tag, which is not recognized by browser. And the second empty facet value is with half-open "<No Position defi..." also treated as malformed html tag.

To fix this problem, you have to modify Common\Templates.cs source code file. perform HttpUtility.HtmlEncode  for facet.DisplayName before assign it as link.Text


//Templates.cs source code

void link_DataBinding(object sender, EventArgs e)
            /******irrelevant code omitted*******/     

            if (facet.DisplayName.Length > _cropMax)
                link.ToolTip = facet.Name;

                //HtmlEncode facet display name
                link.Text = HttpUtility.HtmlEncode(facet.DisplayName.Remove(_cropMax)) + "...";
            else link.Text = HttpUtility.HtmlEncode(facet.DisplayName);  //HtmlEncode here too

            /******irrelevant code omitted*******/   

After you re-deploy fixed version of the feature and web part, the search result should now look like


James Tsai .Net VSTO SharePoint Blog MOSS Faceted Search fix encoding problem


Everything look fine now? If you now click on the link you just fixed as followed above steps. You will see this in returned refine search result page.


James Tsai .Net VSTO SharePoint Blog MOSS Faceted Search decoding problem


HTML Decoding


Yes, the problem is obvious. Whatever we encoded, we must decode it. Otherwise Faceted search web part will pass encoded query value to do refine search. And it will not return correct result (or no result at all, since encoded string will not look pretty).

To fix this problem you will need to modify Common\Utility.cs source code file. Do HttpUtility.HtmlDecode for query.Properties[property] before add it to keywords ArrayList.


public static string BuildQueryString(SearchQuery query)
             /******irrelevant code omitted*******/  

            foreach (string property in query.Properties)
                // consider empty facets and don't add them to qs
                if (query.Properties[property] == Common.Constants.UNNAMED_FACET_VALUE) continue;
                ka.Add(string.Format("{0}:\"{1}\"",property, HttpUtility.HtmlDecode(query.Properties[property])));

             /******irrelevant code omitted*******/  

            return qs;


If you followed all above steps you should now see correct refined search keyword and results.


James Tsai .Net VSTO SharePoint Blog MOSS Faceted Search decoding problem fix


SPFile.CustomizedPageStatus & SPFile.RevertContentStream - The way to check SPFile Ghost/Un-Ghost status and Revert to site definition via code

In my previous post I've discussed how to modify page layout properties through code (in feature activated event). One thing to note is that in the last part of code sample, it requires you to perform following steps to get page layout properties updated.

1. Check out page layout
2. Call page layout .Update()
3. Check in page layout

What I didn't talk about in the post is that after you Check-out the page layout file and check it back in, the actual file gets un-ghost from server.

(If you don't know what ghost / un-ghost are, there are many great articles you can find and read about it over Internet)

Now let's examine closer to see what happens to the layout files after code execution.

Before check-out

James Tsai .Net  SharePoint VSTO ASP.Net blog - MasterPageGallery page layouts

Above displays all page layouts available to team site by default.

Here we check each page layout Ghost status without any modification made to the file:

using (SPSite site = new SPSite(siteUrl))
    using (SPWeb web = site.OpenWeb())
        PublishingSite publishingSite = new PublishingSite(site);
        PageLayoutCollection pageCollection = publishingSite.PageLayouts;
        foreach (PageLayout layout in pageCollection)
            SPFile currentFile = web.GetFile(layout.ServerRelativeUrl);

            Console.WriteLine("{0} --before checkout-- {1}",layout.Name, currentFile.CustomizedPageStatus);


James Tsai .Net  SharePoint VSTO ASP.Net blog - ghosted page layouts

You can see that CustomizedPageStatus  is "Uncustomized" means that the file is in Ghost status.

After Modify and Check-in

Now, let's try update each page layout's description text and check it back in, then we check its ghost status again.

foreach (PageLayout layout in pageCollection)
    layout.Description = "James.Tsai.Net -"+layout.Description;
    SPFile currentFile = web.GetFile(layout.ServerRelativeUrl);

    Console.WriteLine("{0} --after update-- {1}",layout.Name, currentFile.CustomizedPageStatus);


All descriptions are updated

James Tsai .Net  SharePoint VSTO ASP.Net blog - MasterPageGallery page layouts description

But if you check all layouts Ghost status again, you see they are now in un-ghost status (Customized)

James Tsai .Net  SharePoint VSTO ASP.Net blog - unghosting page layouts

Make it ghost file again

To set them back to ghost status, we can use "Reset to site definition" option in site setting page. If you don't want to do it from UI, SPFile also provided a method for you to do it from code.


foreach (PageLayout layout in pageCollection)
    SPFile currentFile = web.GetFile(layout.ServerRelativeUrl);

    Console.WriteLine("{0} --after revert-- {1}", layout.Name, currentFile.CustomizedPageStatus);

James Tsai .Net  SharePoint VSTO ASP.Net blog - reset to page layouts definition

For all the templates file & page layouts, they should always in ghost status for both performance and consistency across sites. If you know somewhere in your code breaks ghosting, don't forget to revert it back to its ghost status. You can always check its status by calling SPFile.CustomizedPageStatus().


How to update properties of an already provisioned and in use page layout

Let's say you have a custom page layout and you want to give this page layout an unique Publishing Preview Image. To allow users to easily choose the right page layout among many different layouts.

This can be done easily if it is a NEW page layout. All you need to do is to define the PublishingPreviewImage element when you provisioning it.

<!-- PageLayout.xml -->

<Elements xmlns="">

  <Module Name="PageLayouts" Url="_catalogs/masterpage" Path="PageLayouts" RootWebOnly="TRUE">

<File Url="CustomPageLayout.aspx" Type="GhostableInLibrary" IgnoreIfAlreadyExists="TRUE"  >

      <Property Name="Title" Value="My Custom Page Layout" />

      <!-- new added description -->

      <Property Name="MasterPageDescription" Value="This page layout is used for new sites." />

      <!-- new added preview image -->

      <Property Name="PublishingPreviewImage" Value="~SiteCollection/_layouts/images/pagelayouts/custompagelayout.gif" />

      <Property Name="ContentType" Value="Custom Content Type" />

      <Property Name="PublishingAssociatedContentType"

                Value=";#Custom Content Type;#0x010100C568D1D2D3D0A14D9B2FDCC96666E9F2007123456EC3DB064584E219954237AF123456;#"/>




Problem -

What if you want to do this for the page layout which already in use and provisioned?  The challenging part is that when page layout has already been provisioned and used across webs. Any metadata changes made in page layout provision feature will NOT update its metadata in SharePoint site.

That means even if you changed PublishingPreviewImage element value. The preview images for the already in use page layouts in CreatePage.aspx will still use default image (DefaultPageLayout.png)


This screen shot shows the page layout is still using default preview image

James Tsai .Net SharePoint Blog - Page Layout Preview Image Default 


Solution -

Create a SPFeatureReceiver class and register it with same feature that provisions page layout(s) . In the FeatureActivated, update each page layout metadata base on the page layout manifest file.

Every time you want to update custom page layout property, just update manifest file, deactivate and re-activate this page layout provisioning feature.

This screen shot shows after reactivated the feature page layout is now using new preview image and new description.

James Tsai .Net SharePoint Blog - Page Layout Preview Image Default

Note: Although you can update each individual layout manually , you would expect the update should be done automatically every time you change layouts manifest and provision them again. That is the main reason to warp the logic inside feature receiver class to mimic the automation.


Please see below for code samples

<!-- Page layout provision feature schema -->



          Title="Custom page layouts provisioning feature"

          Description="Provisions custom page layouts."



          Hidden="FALSE" DefaultResourceFile="core"           


          ReceiverAssembly="James.Tsai.Net.SharePoint, Version=, Culture=neutral, PublicKeyToken=12345bf123eec123"




    <ElementManifest Location="PageLayouts.xml"/>



The feature receiver class will look like following code sample.

//Feature receiver class

class PageLayoutProvisioningFeature : SPFeatureReceiver
    public override void FeatureActivated(SPFeatureReceiverProperties properties)
        using (SPSite site = (SPSite)properties.Feature.Parent)
            //Load manifest file as XmlDocument.
            XmlDocument manifestXml = LoadPageLayoutsManifestAsXml(properties);

            //Load all page layouts for the site into dictionary
            Dictionary<string, PageLayout> allLayouts = LoadSitePageLayouts(site);

            //Now we have an entire page layouts manifest as XmlDocument.
            //We also have a collection of all page layouts
            //We can now step through each page layout defined in manifest and check every propertie
            //against layouts in allLayouts Dictionary.

            //Update layout property in allLayouts Dictionary with value defined in manifest file

            //Check out page layout
            //Call pagelayoutsDic[pagelayoutName].Update()
            //Check in page layout

    //LoadPageLayoutsManifestAsXml(){} ... see next section

    //LoadSitePageLayouts(){} .... see next section

LoadPageLayoutsManifestAsXml method

private XmlDocument LoadPageLayoutsManifestAsXml(SPFeatureReceiverProperties properties)
        //Get feature definition as xml node
        XmlNode def = properties.Definition.GetXmlDefinition(System.Globalization.CultureInfo.CurrentCulture);
        //Create xml namespace manager
        string wss = "wss:";
        string xpath = wss + "ElementManifests/" + wss + "ElementManifest";
        XmlNameTable nameTable = new NameTable();
        XmlNamespaceManager manager = new XmlNamespaceManager(nameTable);
        manager.AddNamespace("wss", "");
        //Create an enumerator and get ElementManifest node
        IEnumerator enumerator = def.SelectNodes(xpath, manager).GetEnumerator();
        XmlNode current = (XmlNode)enumerator.Current;

        //Get manifest file name PageLayouts.xml from ElementManifest node
        string PageLayoutsManifestFileName = current.Attributes["Location"].Value;

        //Load manifest file to XmlDocument object
        XmlDocument doc = new XmlDocument();
        doc.Load(feature.Definition.RootDirectory + "/" + PageLayoutsManifestFileName);

        return doc;


LoadSitePageLayouts method

private Dictionary<string, PageLayout> LoadSitePageLayouts(SPSite site)
        PublishingSite publishingSite = new PublishingSite(site);               
        PageLayoutCollection pageCollection = publishingSite.PageLayouts;
        Dictionary<string, PageLayout> pagelayoutsDic = new Dictionary<string, PageLayout>();
        foreach (PageLayout layout in pageCollection)
            pagelayoutsDic.Add(layout.Name, layout);


        return pagelayoutsDic;