File Modified date misalignment in SharePoint, Explorer View and file shares/drives

04/09/2012

Most of you have been using for ages now the Explorer View of a Document Library. It’s indeed one of the most controversial features SharePoint (sometimes painful for IT Pros) offered since early days but is arguably one of those feature that seduced the “normal” and non-IT users thanks to the easiness of move/copy drag and drop.

During a recent scenario we discovered a misalignment between a file’s “Modified SharePoint metadata field and the actual Date Modifiedof the file as displayed in the Explorer View (if one uses the Explorer View of a Document Library or if the Document Library is mapped as a network drive on the users computer – very common scenario for users) or if one checks the date of the file properties in Office applications (e,g Word, Excel, etc)

The complete scenario is as follows: the “Modified” column in SharePoint has been basically overwritten during a copy/move operation performed using the Document Library Explorer View and now reflects the “Modified” date in which the moving operation was done instead of the last modification date of the file itself.

Below the picture displays the Modified date as it appears in SharePoint (2/08/2012) which is indeed the date where the file was moved from the source to this Document Library:

Below the picture displays the Modified date as it appears if we use the Explorer View on the same Document Library (with the original date when the file was truly modified (27/7/2012):

When you upload a document to a document library SharePoint will use the last modified date as the date which the upload was done while the Explorer view (Windows) will display the modified date stored as property of the file itself!

This behavior can also be explained from another angle: “Modified” date in SharePoint is actually the Content Type modification date while “Modified” date in Windows (classical file shares or drives) is the date of the file itself.

Well, in our case this behavior creates big problems given the fact that many users continue to use the Explorer View in order to access and find their work files in their department or team site (most of them simply remember that the file they look for it was modified at a certain date in time so they will look for that file rather sorting on the modified date than on performing a search).

We looked than for a way of mapping these inner file properties reported above, carrying the last modification date of the file, onto a site column to be shown in a Document Library View. We know that this approach is very useful for search purposes: in Shared Services Administration -> Search Administration -> Metadata Property Mapping we can map the properties Basic:14(Date and Time), Basic:16(Date and Time), ows_Modified(Date and Time) onto a new property labelled, for instance, LastModifiedTime (see pic below).

Nevertheless, this mapping seems to be useful for search purposes only and unfortunately Microsoft confirmed that there is no out-of-the-box configuration for our purpose.

It seems that the only out-of-the-box solution would be to use Backup and Restore which will then preserve the original last modified dates…or custom development of course (event handlers, etc)


Mobile applications vs Web Mobile

30/03/2012

The corporate question often arise these days:  should I build a mobile web site or a mobile application? Well, there are already thousands or millions of pages on pro and cons but below, thanks to the feedback I got attending the Usability Week by NNGroup, I tried to summarise the essence of this discussion:

  1. MOBILE APPLICATIONS: For highly specialized services and/or very specific content with frequent recurrent users a mobile application makes sense. Nevertheless, marketing/service positioning can be a special and valid reason for building a mobile application for the organization. (e.g. I need to have an app because my concurrence has already or is planning one.
  2. MOBILE WEB: For all the rest(kinds & targets of users, the accessibility is the most important requirement so a mobile website is enough.There are nevertheless (at least) 2 main technical implementation options:
  • Keep your web site as it is but place a specialised proxy in front of it so it detects when a mobile is coming to your web site. You can easily outsource the detection and service of mobile web pages (reverse proxy or similar solutions to Adesso in Germany) as it will be impossible for you to maintain the caracteristics of each and every main mobile device out there.
  • Build a mobile version of a site using for instance adaptive layout (one size that fits all – average quality; see Lessframework exemple : http://www.lessframework.com/). Device detection layer (point above) can still be outsourced or insourced.

But ultimately DO NOT FORGET this: Building a Mobile Application or a Mobile Web Site…..this MUST BE a business decision and not an IT one!



Developing Windows Phone 7 Applications for SharePoint 2010 (MDV08)- mynotes

01/10/2010

SharePoint Connections 2010, Den Haag, 29 September 2010

Session: MDV08: Developing Windows Phone 7 Applications for SharePoint 2010 (MDV08)

Speaker: PAUL STUBBS (http://blogs.msdn.com/b/pstubbs/)

  • Globally Web parts are not rendered but only content is rendered on the mobile from a SharePoint site.
  • If you wish to render web parts content, use Mobile web part adapters:
    • Renders alternate view
    • Lookup in browser.compat config file
    • Supported in SharePoint Foundation
    • Use Microsoft.SharePoin.WebpartPages.MobileAdapter
    • Override CreateControlForDetailView()
  • Applications supported on Windows Phone:
    • Silverlight (business applications)
      • Modern XAML/event driven application UI framework
    • XNA (games)
  • Capabilities of the Phone (features writable against SharePoin)
    • Input
      • Touch
      • Hardware buttons
    • Media
      • Digital media capture&playback
      • Media Library access
    • Data
      • Isolated Storage
      • LINQ
    • .NET
      • Superset of Silverlight 3.0
      • WCF (SOAP and REST services)
    • Phone Access
      • Integrated access to phone UI
      • Sensors, camera, microphone
      • Picker for contact and photos
    • Integrated with could Services
      • App deployment & updates
      • Notifications (very important for mobile services)
      • Location
      • XBOX live
  • Authentication to SharePoint is an issue for development on mobile phones
    • The phone DOES NOT SUPPORT windows authentication yet but only Forms based authentication
    • Forms based authentication
      • Supported very easily through claims provider in SharePoint 2010
      • Provider neutral: SQL, LDAP
      • Solution: forms authentication provider can be set to LDAP and so the users will be still use the same username and password as the one in active directory
  • Windows Phone tips for SharePoint
    • Only supports Forms Based Authentication sites in SharePoint
    • FedAuth Cookie is HTTPOnly
    • Pass along in a CookieContainer: set enableHttpCookieContainer on binding. One you have it (after authentication) always pass your CookieContainer with any list call you initiate.
    • Authentication.asmx not compatible with Service Reference in Visual Studio: use instead HttpWebRequest
    • Access to list web services works fine.
  • Phone tools are working for Windows 7 only – phone development machine needs to be on Windows 7 (if you start the phone emulator in a server it will reboot L )
  • Check slides for a development environment schema
  • Tooling
    • http://Developer.windowsphone.com download all tools and emulators
    • Download SharePoint 2010 Developer VHD (Windows Server 2008 R2 Standard, SharePoint Enterprise with Fast, etc)
  • Run your App on a Phone for tests: Matthijs Hoekstra (matthijs.hoekstra@microsoft.com )

Creating Content-Centric Sites with SharePoint 2010 Web Content Management (MNC04) – my notes

30/09/2010

SharePoint Connections 2010, Den Haag, 29 September 2010

Session: Creating Content-Centric Sites with SharePoint 2010 Web Content Management (MNC04)

Speaker: ANDREW CONNELL (http://www.andrewconnell.com)

  • Planning a SharePoint 2010 WCM Site
    • Define major areas of the site
    • Define the types of content
    • Define different rendering options on the site
    • Define major actors for different sections
    • Define content review & approval process
    • Define content management plan
  • Tooling small nice application from CriticalPath Training company (check on the Members area of the web site for it: AC’s SharePoint 2010 Site Collection Keep Alive Job Utility (v1.1 – Updated June 7, 2010) for site warm-up : to be used before demos and presentations or for testing purposes.

Creating a Rich Business Application with the Managed Client Object Models in SharePoint 2010 (MDV01) – my notes

30/09/2010

SharePoint Connections 2010, Den Haag, 28 September 2010

Session: Creating a Rich Business Application with the Managed Client Object Models in SharePoint 2010  (MDV01)

Speaker: Andrew Connell (http://www.andreconnell.com/blog)

  • This session contained mostly things that apply to SharePoint 2010 but there were also remarks about SharePoint 2007
  • Important points
    • Overview of building Rich Internet Applications (RIA)
    • Options we have in SharePoint 2007
    • Options we have in SharePoint 2010
  • Riche Internet Applications (RIA)
    • As a definition, RIA engages client-side solutions, meaning AJAX, Flash, Silverlight
    • In SharePoint 2007 you might need extra setup and config
    • In SharePoint 2010 it’s out of the box + extra connection points from client applications.
    • Security trimming is present in both versions -> nothing to do for the developers
  • Option we have for RIA In SharePoint 2007:
    • ASP.NET 2.0 (. ASMX web services)
    • WCF (.SVC) technically not supported
    • Silverlights v3 which requires ASP.NET 3.5 and a lot of edits in web.config
  • Options we have for RIA in SharePoint 2010
    • Better consume & create web services (ASP.NET & WCF)
    • Client Object Model
      • A lot more interaction is possible with the site collection
      • Extends the SharePoint object model to Silverlight
      • Enables developers to work with familiar objects rather then web services
      • It’s centered on the ClientContext  – handles communication with SharePoint sites.
      • Sends commands to SharePoint (ExecuteQueryAsync(success,fail))
      • Using it requires 2 references:
        • Microsoft.SharePoint.Client.Silverlight.dll; (this includes .NET Client OM API’s
        • Microsoft.SharePoint.Client.Silverlight.Runtime.dll (does all the communication to SHAREPOINT server back and forth); (to be found in […]/14/templates/layouts/ClientBin/)
        • LINQ is only working on servers side…on client side you will need CAML query!..CAML is dead..long live CAML J
        • Works only using async communication
    • OData/WCF Data Services / RESTful services
      • Very easy to read/write data to lists and sites.
      • It does not allow changes of structure, permissions, or navigation like within the client model.
      • OData is an open protocol founded on REST principals
      • WCF Data Services is the Microsoft implementation of OData open protocol
      • Does not work out of the box!! J To make it work you need WCF Data Service Update for .NET 3.5 SP1
      • OBS: Dataaccesspolicy.xml defines who can call which service (normally found in the root of the Inet application)
    • Sandbox Solutions
      • Code deployed by non-admins via browser
      • Runs in an isolated environment

Business Connectivity Services Deep Dive (MDVWRK1) – my notes

30/09/2010

SharePoint Connections 2010, Den Haag, 27 September 2010

Session: Business Connectivity Services Deep Dive (MDVWRK1)

Speaker: Scot Hillier (scot@shillier.com, http://hillier.codeplex.com)

 Key ideas covered by this seminar:

  • External Content Types (ECT)
  • NO code BCS solutions
  • BCS architecture, BCS Solution features and service application
  • Information in this seminar applies mostly to SharePoint 2010 (and Office 2010 for client applications)

 Notes

  • Information Workers = people turning on Outlook when they walk in their office; 
  • As solution providers we should ask ourselves: Are we helping really the users (information workers)?
    • Map net drives – creates masses of useless documents
    • Build asp.net applications – generally work on structured data..which is rather not the case for users.
    • SharePoint sites – traditionally works with info in lists in the content DBs; web parts don’t work with unstructured data in documents
  • Client tool for BCS applications: Office Outlook is seen as the primarily client tool for BCS applications development. SharePoint Workspace is much less as it is not customizable or very little.
    • Moving a ASP.NET to SharePoint scenario: copy the ASP.NET application to _layouts directory and it works but there is 0 integration: Data will not be indexed so it will not be present in search results
  • BCS lists have built in throttling (retrieval limited to 200 items ). Through configuration we can increase this limit (see below)
  • BCS server side features (help you to build applications like in ASP.NET):
    • External lists
    • Profile pages (present a single record from the DB – record view)
    • Actions
    • InfoPath Forms
    • External data web parts & Columns (in MOSS 2007 language they were called Business Data instead of External Data)
    • Search Integration
    • User profiles supplements
    • Custom solutions
  • BCS client side features (requiring Office 2010)
    • Office Outlook features (take info online)
    • SharePoint Workspace
    • Office Word External Data Column Integration (!!!)
    • Custom solutions
  • Not all BCS features are available in SharePoint Foundation. Check the slides for a list of BCS features SharePoint version.
  • BCS has a full API (we can talk to all objects client or server from BCS)
  • Varions types of solutions that we can build using BCS:
  • External Content Type (ECT) is the most important concept in all of BCS
    • It defines the columns of a particular entity
    • It maps the system from where it  takes the values
    • It defines the operations we can perform
  • The definition of ECT is FARM Wide
  • Connectors in BCS:
    • There are 4 but only three displayed: .NET, SQL, WCF. The Forth is ‘Custom connector’. What’s the difference between a  .NET and when is ‘Customer connector’?:
      • .NET connector = Connectors for schemas (e.g. DB Schemas or Exchange object model) which do not change (.NET built by people in house for internal systems)
      • ‘Customer connector’ = Connectors for schemas (e.g. DB Schemas or Exchange object model) which change in time (built by third parties in order to be sold .. more generic = ‘Customer connector’)
  • Steps to build a typical BCS Solution without code:
    • Define an External Content Type (ECT) using the SharePoint Designer
      • Define the connection using one of the available connectors
      • Define the operation (there is a wizard for operations and field mapping). OBS: You must define at least READ LIST and READ ITEM operations so that you can create an External list in SharePoint for use of this ECT later on.
      • Create an Eternal list in SharePoint and use the newly created ECT.
      • You are DONE!
      • Optional: create an office Add-in directly from the External List (VSTO package) so that you can use that data in your office application (like an external list of contacts for example). Furthermore, you can take that data offline with you and re-synchronize it when you go back online.
  • BCS Authentication (check slide):
    • Passthrough,
    • RevertoSelf (is disabled by default and can be activated with a powershell command),
    • WindowsCredentials,
    • rdbCredentials (for DBs connects using non Windows credentials returned from the SSS),
    • Credentials (for web services connects using non Windows credentials returned from the SSS)
    • IMPORTANT slide to show the connectors vs SharePoint2010 Authentication Scheme. Combinations of authentication schemes. 90% of BCS problems are found
    • BCS connection to data: NTLM vs Kerberos discussed for connections to database properties: when establishing a connection, connecting using the option ‘users’ identity’ doesn’t really work in most cases (especially on enterprise production systems with separated servers or for NTLM as there are two hops between the user and the sql data). Using Impersonated Windows or Custom Identity requires Secure Store Application ID
  • SharePoint Designer
    • Becomes with 2010 version a critical tool especially for building BCS solutions
    • Allows you to use ‘Generate all operations’ on a table and it will generate all CRUD operations. A wizard will help you map various fields for all these operations.
  • BCS Architecture:
    • On server side: BDC (Business Data Connectivity) Service Application has it’s own database for ECT content (db dedicated to this service application); + BDC Server runtime. On top of it we find the Secure Store Service (SSS),  the search, workflows and Web parts of SharePoint and finally the VSTO package and External Lists on the SharePoint Site level.
    • BDC services on client side includes the BDC client runtime and the client cache (allowing the client to take the date offline)
    • When deploying a VSTO package, the Client gets a proxy for the connector as well
  • Key LIMITS of external lists in SharePoint
    • You can not initiate an workflow from an item in an external list but you can use an item in an external list in a workflow initiated on a SharePoint list
    • Does not support event handlers
    • Using BDC server runtime can help in overcoming external lists limitations! – Custom web parts.
  • Key Throttling limits (default, maximum) for BCS:
    • Connections (global scope): 100, 500
    • Items (SQL scope) / rows returned for a query: 2k, 25k
    • Connection timeout (WCF, SQL scopes): 60s, 600s
    • Size of data (WCF scope): 3Mb, 150Mb
    • [Photo IMAG0499]
  • Business Data Connectivity (BDC) permissions:
    • Edit (CRUD operations)
    • Execute
    • Selectable in Clients (pick ECT for use in lists, web parts, etc)
    • Set Permissions (can set permissions)
    • Typical error when using the first time: YOU need to have Execute permissions on the model of data itself
  • THE server never caches the DATA but it CACHES the BCS model. ImportantfFor client/Server BCS solutions…the client needs to reconnect (take the data off line and then take it online again) or ask for a forced Update this list. (This needs to be planned).
  • Business Data Connectivity (BDC) Metadata model is  an xml file defining the External Content Type (Fields, Operations, Relationships, Actions) and the external system itself (server details, connections, etc). This file contains EVERYTHING you might define through the interface for the model. This file is the entire BCS solution.
  • Deploying BCS solutions:
    • The entire BCS solution in contained in the metadata model = xml file.
    • Deployment DEV-TEST-PROD: models can be Exported and imported.
    • SharePoint Designer supports export but no import. Importing is only supported by the Service Application interface (Central Administration).
    • Import/Export can be done using Powershell.
    • Partial Export or multiple file export: Between DEV and PROD environments there might be different permissions or resources (external systems) – the model can be exported in more than one single file (e.g. one with the ECT definition, one with the external systems definition and another one with permissions)
    • The Entity element has versioning!!! You must increase the versioning number when importing in PROD
  • When developing BCS applications to run on Client and Server you have to plan:
    •  Authentication (no secure store on the client side)
    • Updates (if the model change on the server it does not change for the client unless a forced Update the list is made or resync initiated (data offline and then online again) (see above the caching of model on the server level)
  • ‘.Bdcm’ files (exports of a BCS metadata model) are recognized by Visual Studio which proposes a certain number of tooling for editing the file but.. The tooling in Visual Studio only works with .NET assembly connectors! As an alternative,  the file must be edited as a simple normal xml file. (good XML knowledge is a must for a BCS solution developer)
  • BCS Data filters: SharePoint Designer does not support all types of filters. you should go and edit the XML Model file if you need a specific one. Check slide with filters.
  • Relations between more External Content Types – ECT associations are possible:
    • One to Many
    • Self Referential
    • Reverse
    • Many to Many, foreign keyless (this is supported in the model BUT not in SharePoint Designers
  • Association Navigator: able-ing the user to define relations between more ECT. Using the Association Navigator allow SharePoint to generate some UI. For instance if we defined the association (parent/children), on the profile page of the parent will have displayed his related childs.
  • ‘Profile Page’ a page containing all the fields of the given ECT. Once defined, it becomes the default action for the record.
  • Design and Presentation: Improving the UI for out of the box BCS solutions could be done through:
    • ASPX forms
    • Infopath forms
    • Format data by the external system before it-s fetched by BCS or create a database view or a web service that returns the formatted data (for instance as data only is not supported you could build a stored procedure that casts the datetime field to date only formatted as you wish to be displayed finally on the page)
    • Use xslt with External Data web parts
    • Create custom field type and field control.
    • Create a custom Web part that uses BDC runtime API
    • Create a .NET assembly connector that returns formatted data
  • Key REMARKS:
    • dateTime type of data is mapped to External Lists!
    • Date only not supported!
    • Integer64 not supported
    • Default pages (DispForm.aspx, EditForm.aspx, NewForm.aspx) use the ListViewWebPart wich you can not extend (no XSLT, no modification of ui possible). By creating a new page (e.g. MyNewForm.aspx) it will use the DataViewWebPart which is extensible and accepts XSLT for display of data.
  • SharePoint Designer 2010 feature: save site as template will save it as a solution package .wsp!
  • BCS Solutions Client Side: Client Data Synchronization:
    • Click-once deployment package: package is created in ‘ClientSolution’ folder
    • Client cache – things to know:
      • It Is on Client side, per user, in memory, SQL CE database!!
      • Client caches the data and the model! Remember that the server only caches the model, never the data
      • There is as well an ‘operation queue’
      • BCSSync.exe
        • Calls the ‘Finder’ method to populate the cache
        • ‘SpecificFinder’ method is necessary to complete cache population
        • Client CACHE considerations: There is no way of deleting the data in the cash…but there is a garbage collection for stale data (compare the old data with the new data at the end of cache expiration time and flags the old data as ‘ garbage’ which ultimately will be deleted but the entire operation is 100% self managed by the BCS and not by you as user)
      • PLANNING: Plan for client/server calls (especially if you have parent child relations when you could end up with many many calls)
  • BCS Solutions for Microsoft Outlook
    • We can create declarative solutions:
      • Custom views
      • Custom form regions (see below for definition)
      • Custom ribbon elements
      • When deploying VSTO add.ins solutions to Outlook, Outlook creates a ‘REGION’ for fields he does not understand from the model. It’s good to know that Custom Form Regions are created direct in MS Outlook 2010 (saved in the .ofs file)
      • Tooling:
  • 4 Object models for BCS
    • BDC Server Runtime (Microsoft.Sharepoint.dll, Microsoft.BusinessData.dll)
    • BDC Client Runtime (Microsoft.Ofice.BusinessData.dll, Microsoft.BusinessData.dll
    • Client Cache object model
    • Administration catalog object model
    • OBS: Client and Server object models are essentially identical depending on the catalog you access.
    • Using BDC server runtime can help in overcoming external lists limitations! – Custom web parts.
  • BCS solutions cannot run in sandbox solutions. (Requires more trust)
  • Demo contains an app in console mode accessing the data catalog and a web part calling a ‘Finder’ method and ‘SpecificFinder’ method.
  • .NET Assembly connectors
    • .NET Assembly Connector is supported by tooling in Visual Studio (tooling to create the XML model file: BDC Model Explorer, Method details, Properties pane, etc): in Visual Studio choose ‘Business Data Connectivity Model’
    • Two classes are mandatory for a .NET assembly connector: the services class and the entity class
    • They are to be deployed always as FARM level features
    • Needs full trust to run (the assembly is deployed in the GAC)
  • SEARHC: enable search on the BCS model:
    • Modify the LobSystemInstance element of the model and find the properties:
      • RootFinder – it tells search which Finder to call in order to index things
      • ShowInSearchUI – makes this model appear in Search Service Application UI (listed as Business Data in Content Sources); there is the possibility to create a new Scope and then have the search performed only in this scope (e.g. a particular ECT interesting for the company or for a department)
      • Security applied on search results (generic):
        • Crawl time trimming only works with windows compatible external systems
        • Query time trimming might be very resource consuming
  • SOA Architecture hint: rather develop the Web service returning the desired data on the External System and use WCF to connect it to SharePoint.