Category Archives: SharePoint

SharePoint Announcements from Microsoft Ignite 2018 – User Group Presentation

Thanks for everyone that came along to the Sydney SharePoint User Group this month. It was great to be able to deliver so much exciting SharePoint news following all the announcement made at Microsoft Ignite. Given Microsoft Ignite now covers far more than just SharePoint it takes a while to distil the SharePoint specific announcements from over 700 sessions that were presented over 5 days at Microsoft’s biggest conference of the year.

I’ve kept the presentation to just the User/IT Pro announcements (sorry developers I couldn’t fit all the news into a 1 hr presentation!)

Feel free to take this presentation and use it for your own user groups or internal within organisations.

SPUG-MSIgnite-Announcements

Open slide deck: SharePoint-User Group-Sydney-Microsoft-Ignite- 2018-SharePoint-Announcements-Slides-Deck

Microsoft Ignite 2018 – Office Developer Announcements

As the dust settles on Microsoft Ignite for another year I’m left going back over my notes and recalling discussions I had for all those key announcements, advice and snippets of gold that will have a real impact for Office developers.

If you are looking for a high level list of announcements made at the conference, the Ignite Book of News is a good place to start although it doesn’t cover many of the announcements that were made in the Office Developer area – this book covers a lot of the Azure announcements, which most Office developers will have a mild interest in (we have to host our code somewhere!)

Here’s some of my favourite announcements:

Do more with new upcoming SharePoint development capabilities announced at Ignite 2018

  • Call Microsoft Graph and Web APIs and deploy Extensions across your SharePoint sites
  • Deploy your web parts and application pages to Microsoft Teams
  • Connect across components with dynamic data capabilities
  • Deliver complete applications with application pages
  • Harness more of SharePoint with new Microsoft Graph APIs

Microsoft Graph @ Ignite 2018

  • Managed access to Microsoft Graph (data connect to bulk export to Azure subscription)
  • Notifications API
  • Dynamics is now in Microsoft Graph
  • New PowerApps templates
  • Security API
  • Microsoft Teams, Messages, Calendars, Files, and Folders

Microsoft Authentication Library (MSAL)

  • In preview but suitable for production use
  • Capable of reaching both v1 and v2 services
  • MSAL JavaScript library for serverless implicit flow scenarios

I thought this years conference was very well run and the volume of people moving about the conference centre wasn’t overwhelming. I had a lot of fun meeting new people and reconnecting with old friends. It’s great to have such knowledgeable Microsoft staff accessible on the expo hall floor (both from a Marketing and Engineering side) to discuss particular scenarios, technologies, ad bounce ideas off.

 

Allowing different Azure AD app registration permission sets for a single app (user and elevated admin consent) using the v1 auth model

With Azure Active Directory Application Registrations there are two versions of authentication model available.

v1 – all the permission scopes that your app may require must be consented to by the user up front.

v2 – permission scopes can be asked for dynamically as your app is running, if the user hasn’t already consented to the required permission scope then they will be challenged for consent at that time

v2 is a far more flexible model as it allows users try out and/or start using your app without having to consent to everything your app could ever want to get access to. After getting comfortable using the app, as the users explore and use more specific and advanced features in your app, you can ask for further permissions. Even more advantageous, certain API calls require a tenant administrator to consent to the permission on behalf all users. With v2 auth model end users can use features of the app that they have authority to consent access to, and then if a admin consents to some of the admin only permissions then even more features could be lit up in your app.

Here’s a good rundown on the state of app registration and auth in a recent episode of the Microsoft Cloud Show and you can read about the difference between v1 and v2 auth in the official Microsoft docs.

The Problem

Not all backend service APIs support the v2 auth model, and you can’t mix and match v1 and v2 auth model. If one or more of the backend service APIs you require only supports v1, then your entire app (and access to all service APIs) will be done using v1. At the time of writing the Microsoft Graph supports v2 auth, but SharePoint only support v1 auth. There is a technique for taking an refresh token acquired using v2 auth and exchanging it for a SharePoint access token but this technique can only be used from a custom Web API, and not from a Single Page Application (SPA) as it’s not safe to expose a long lived refresh token in client side code (i.e. JavaScript running in the browser).

This means there’s situation where you are stuck with v1 auth for now. Under v1 auth if your app has at least one permission that requires admin consent, then ordinary users are not going to be able to simply start using your app on there own, we are back to the days of having to “go through IT” to have an admin approve the app before it can be used.

The Solution

Well, it’s not a silver bullet solution that is going to fix any scenario, but the technique I’ll discuss here allows you to define two sets of permissions for your app. One set of permissions that contains just the minimal set of permissions to get users started using your app (you wouldn’t want any permission that require admin consent here) – We’ll call this the User Permission Set, then a second set of permissions (that contains those tougher to get approval for permissions that require admin consent) – We’ll call this the Elevated Permission Set.

What we are aiming for is the app to run with just the restricted User Permission Set (so that anyone can quickly start using your app) but maybe not with all the features enabled, and then allow an administrator to optionally provide consent on behalf of all users which then allows the app to use the Elevated Permission set (for all users).

I’ll assume you have already been able to create an app that successfully authenticates and consents a user against a single permission set (here’s a good starting point with Azure authentication concepts if you aren’t to this stage yet)

Step 1 – Create an Application Registration per permission set

Create an application registration for both the User Permission set and the Elevated Permission set (this will be a superset of the User Permission set). These registrations should almost be identical (e.g. same Reply URLs), but they will have different Application IDs, and obviously different permissions to represent the different permission sets. We will call these the User App Reg and the Elevated App Reg.

Step 2 – Change the normal auth flow to try to acquire tokens using the Elevated App Reg first

Your normal auth flow would be to try to acquire an access token for the service endpoint specifying the App Reg Id. Now we have two possible App Reg Ids, so what we do is that we try to acquire the access token first using the Elevated App Reg Id. If you are able to get the token then you are away just like the normal app flow (in this case consent must have been granted by an admin previously). But here’s the trick, if you fail to get the token (and the reason returned is that you need to prompt for consent) then proceed with your standard flow to acquire the token this time using the User App Reg Id and prompting for user consent if required. This way the user is able to start using your app as they will have authority to consent to the User App Reg.

Step 3 – Track which App Reg Id is in use

Once this auth flow is complete, track in the state of your app which App Reg Id you successfully acquired the token for, as that token will only work with the App Reg Id used to acquire it. Example: if the call to acquire the token using the Elevated App Reg Id worked then all future calls should specify the Elevated App Reg Id.

Step 4 – Conditionally protect features that require the Elevated App Reg

Now you are tracking which App Reg is in use you will know when your app only has the restricted User Permission Set. You can use this to hide features or prevent them from being used.

Step 5 – Expose a way for administrators to provide admin consent

Somewhere in your app you can provide the ability (e.g. a button) for a an administrator to provide admin consent. This will just launch the prompt for admin consent login URL and (always use the Elevated App Reg Id for this). Now when a user tries to use the app (see step 1) the attempt to acquire the token using the Elevated App Reg Id should work since an administrator has provided the consent.

If you are feeling really awesome you could (in the same session of your app) go through your auth logic again without restarting the app the discard the tokens you will have acquired against the User App Reg and get new tokens now against the Elevated App Reg and light up those new feature of your app immediately.

Video example of an Outlook Add-in utilizing this technique to provide user and elevated permission sets within a single add-in and allowing an admin to dynamically provide consent enabling additional features.

 

 

 

Wonder what a Microsoft 1:1 hackathon looks like? OnePlace Solutions Teams/Graph Engagement Experience

microsoft-teams-logo.jpg

I was fortunate enough to be involved in a 1:1 hack engagement with Microsoft recently where OnePlace Solutions hosted some eager Microsoft engineers for a week long engagement. The intention was to see how we could harness some of Microsoft’s new Teams extensibility options and the Graph API, and for Microsoft to identify limitations or areas for improvement.

The format of the event:

  • brainstorming possible ideas ahead of the event itself
  • discussion and selection of a few possible ideas
  • splitting up into teams and scoping what were would try to achieve within the scope of the hack
  • working in a compressed scrum process (daily stand-ups, task refinement and retros)
  • present to a wider audience on the last day of the hack to show what had been achieved and the business benefit

It was amazing to see how quickly the Microsoft engineers were absorbed into our development team, brought up to speed with our existing code-base, and starting to deliver functionality.

The real takeaway and reason for writing this article it just to let everyone know what an awesome opportunity these engagements are from Microsoft, a bit of what you can expect and that I highly recommend getting involved if the opportunity arises.

What did I see as the biggest benefits to our business of doing this hack with Microsoft?

The tips, tricks and work pattern knowledge sharing that occurred only comes when you truly try to work together on a project and aren’t just academically sharing knowledge. We all work in different ways and by running the hack almost as a true project (in a condensed form) there is a lot more than just the coding that is being discussed. VSTS, scoping, work item tracking, design white-boarding sessions, daily stand ups, retros, git source control, review of pull requests. All this is outside of the actual coding and using the technology being hacked on, but it is also a critical piece of developing in an efficient, scalable and measurable way.

Accelerated and focused learning on new technologies. The speed of getting across where a technology like Teams extensibility is up to, what’s possible when applying to problems we are trying to solve, and that hard first mile of understanding the frameworks, dependencies, and tooling to get the first hello world skeleton running.

Outside of the technology it’s a great opportunity to meet and build relationships with people who share a similar passion and spend a lot of their time working to solve similar problems. At OnePlace Solutions we are a passionate bunch of technologists that enjoy working in a social and supportive environment – from what I’ve experienced the hack is a perfect match for the way we work, with Microsoft bringing the same mindset, energy and support to the hack. We spent as much time laughing and discussing topics outside of technology as we did on it. At the end of the day we are social creatures and I found the hack was a perfect environment that bought people together with a desire to want to work together on a common goal, to challenge and push each other to do more in a fun and supportive way, and have a good laugh at the same time. Having access to global Microsoft resources to get definitive answers quickly, removed the amount of wasted time and frustration which allowed productivity, enthusiasm and energy levels to remain high.

We dedicate an amount of time each sprint to R&D, which usually involves educating ourselves in what is possible with new technologies and APIs and often going as far as prototyping code to see what’s possible and where the limitations are. It’s hard to imagine a better return on investment than spending this R&D time with Microsoft in the format of a 1:1 hack.

So a huge thank you to the Microsoft engineers, we had a great time and my advice to anyone thinking of getting involved with these engagements is that they can have great value to your team.

SharePoint Conference NA – The photos and my takeaways

SharePoint returning to Vegas, would it work like the days of old? There is certainly something special about the SharePoint community and this conference had that tight knit community feel to it.

Vegas amazes me at the amount of people it seems to just be able to soak up without really skipping a beat, making it the perfect place for hosting a conference. There’s no transport required to venues and endless dining and entertainment options.

We had a really busy time on the OnePlace Solutions booth, a big thank you to everyone who stopped by to see what our latest products are capable of and keeping us busy. We had more quality conversations than we were expecting, and it’s great to hear peoples passion for SharePoint and Office 365 coming through in those conversations.

While there was a lot being announced at the conference, I was happiest to see metadata in SharePoint finally getting some love. I had thought that SharePoint had lost it’s way a little on the metadata front in recent years. Especially in the shift to modern UI for document libraries and lists. I believe it is the metadata that made SharePoint so valuable for building solutions on top of. So I was super excited to see the modern document library webpart receive a big overhaul in it’s support of metadata.

Lists also got a refresh and I think the ability to generate a list (and columns of the right type) by directly importing a spreadsheet is genius. I think is a fairly common work pattern for users to start playing with tabular data in Excel and at some point it becomes valuable to share – providing such a simple way of moving from Excel to SharePoint should drive adoption of SharePoint lists as the central shared location for this data and then provide a wealth options on what can be done with that data once it’s in SharePoint.

I was technically impressed (and surprised) by the augmented reality of SharePoint Spaces and the work that had been done to bring this to the masses. I think it appeals more to the content management space than the document management, file management, collaboration and business workflow process areas I typically work in.

Below is a quick video of my reaction after the keynote.

 

 

SharePoint Conference North America – The Sessions that Interest Me

cameron-dwyer-sharepoint-conference-north-america-mvp-march-2018I’ll be attending the SharePoint Conference North America conference, being held in Las Vegas next week. OnePlace Solutions are a proud sponsor and if you are attending I’ll be spending most of my time at the OnePlace Solutions booth in the expo hall so please come and say hi. I was looking through the sessions today and a few caught my eye. I’ve definitely got a leaning towards developer sessions and these are the ones I’ll be trying to get to.

cameron-dwyer-sharepoint-conference-north-america

6 Tips to Perfecting Your SharePoint Game With Structured Content Management

By Mike Miller

https://sharepointna.com/#!/session/6%20Tips%20to%20Perfecting%20Your%20SharePoint%20Game%20With%20Structured%20Content%20Management

Why I’m interested in what Mike has to say:

While self organising and friction free organising of people into groups to work on cross divisional projects has become the hot trend of the last few years I believe the mainstay of core business solutions built on SharePoint benefit from a properly planned and designed information architecture. Take a company that is centred around running projects, if each project was to self organise and manage the SharePoint content differently it makes reporting and visibility across projects very difficult. If this function is core to your business those projects soon add up and before long you have 100’s or even 1000’s of projects that are all structured and run a little bit differently meaning not only is it hard to get visibility and reporting across projects but also for users who have to know the differences between projects and can’t just switch between projects and work with them in the same way. I see this leading to user adoption issues, frustration and decline in productivity.

 

Angular Elements

By Rob Wormald

https://sharepointna.com/#!/session/Angular%20Elements

Why I’m interested in what Rob has to say:

I develop commercial applications (Office Add-ins) using Angular so I have particular interest in the introduction of Angular Elements. My understanding of Angular Elements is that you will be able to develop a component of functionality (UI + Code) that can be packaged up and reused in any JavaScript project regardless of the framework you are using (or no framework at all). This has great promise since you get the benefit of developing using a strong JavaScript framework like angular, but that component is then portable for any JavaScript developer to use without having a dependency on angular. Pretty cool but I’ll have my ears open to how far off this technology is from being supported in mainstream browsers and if there are shims or polyfills that can make it a reality in the short term.

 

Customizing end-to-end modern experiences in SharePoint Online

By Vesa Juvonen

https://sharepointna.com/#!/session/Customizing%20end-to-end%20modern%20experiences%20in%20SharePoint%20Online

Why I’m interested in Vesa’s talk:

Vesa and the PnP community he leads really is the spearhead of the latest SharePoint developer news. Vesa is also a great speaker who tells the honest story and relates well to the developer and the real world scenarios that they are often trying to solve.

 

Develop and deploy Outlook Actionable Messages for optimal user productivity

Julie Turner

Why I’m interested in Julie’s session:

Actionable messages are a great new extension point to Outlook that can really bring user productivity benefit. With the announcement of support for the Adaptive Cards standard at Build recently this could be a great session to get across the opportunities this brings to the developer.

 

Extending Microsoft Teams

Scot Hillier

https://sharepointna.com/#!/session/Extending%20Microsoft%20Teams

Why I’m interested in Scot’s session:

It’s almost impossible to keep up with the rate of change with Microsoft Teams at the moment. There were lots of announcements at Build including the SharePoint Framework webparts able to be surfaced in a tab in Teams. This session should have plenty of new material and Scot has been a leader in the SharePoint community for a long time.

 

Mastering Azure Functions

Bob German

https://sharepointna.com/#!/session/Mastering%20Azure%20Functions

Why I’m interested in Bob’s talk:

I find myself using Azure Functions more and more for automation and non critical tasks. I’m thinking it might be time to hear from an expert about where Azure Functions are up to and whether using them for prime time commercial product and services is a reality.

 

Microsoft Cloud Show

Andrew Connell & Chris Johnson

Why I’m interested:

I’m a regular listener of the Microsoft Cloud Show podcast – it’s my go to place for the latest news in the SharePoint and Office 365 world. These guys don’t take it too seriously and have a good laugh along the way. Both have a long history with SharePoint and and share their opinions and thoughts openly rather than just regurgitating the news as it’s published.

 

 

SharePoint Server 2019: A First Look

Bill Baer & Hani Loza

https://sharepointna.com/#!/session/SharePoint%20Server%202019:%20A%20First%20Look

Why I’m interested in this session:

While Office 365 and SharePoint online make sense for a lot of businesses, so does SharePoint on-premises and in a hybrid setup. I’m not expecting any huge surprises like wow here’s some new features that we haven’t seen in SharePoint online. What I’m interested in what’s missing when comparing SharePoint Server 2019 to SharePoint Online and what hybrid scenarios/ services/ connectors are being supported to stretch SharePoint 2019 into the cloud.

 

Zero to Hero with Microsoft Graph – real-world lessons from 1B+ api calls

Chris Johnson

https://sharepointna.com/#!/session/Zero%20to%20Hero%20with%20Microsoft%20Graph%20-%20real-world%20lessons%20from%201B+%20api%20calls

Why I’m interested in this session:

Chris has been on the cutting edge of a lot of the new developer extensibility points and APIs in the work he does at Hyperfish.  It is one thing listening to a Microsoft employee discuss a new developer toolset, or API, it’s another to listen to someone who is using the things commercially and needs to provide a rock solid offering on top of it. It’s a similar path to what we have been through at OnePlace Solutions and I’m interested to hear of his experiences and the learning he has taken away from it.

Microsoft Graph API, Throttling & SharePoint Lists/Libraries – HTTP 429 Error Code

When developing against the Microsoft Graph you may find yourself experiencing  HTTP 429 Error Codes now that resource throttling is being implemented in different areas of the Graph.

I came up against a strange and somewhat misleading one this week which is worth being aware of if you are using the Graph to access SharePoint lists and libraries using the /sites/ area of the Graph.

I had a service running which started reporting HTTP 429 error codes. I read through all the latest published documentation to try a figure out how the throttling has been implemented and what the limitations are to see what part of the code could be triggering the throttling. As you’ll find the documentation is very non-committal and mostly serves to justify why there are no specific limits and rather algorithms that dynamically determine the throttling based on a large number of dynamic criteria. All of this sounds really fancy and advanced but is not very helpful when trying to identify what could be causing the throttling issue, or what limit your code is hitting.

Here’s the Microsoft documentation links which are well worth the read:

Microsoft Graph throttling guidance

Updated guidance around SharePoint web service identification and throttling

Avoid getting throttled or blocked in SharePoint Online

(Azure) Throttling pattern

Most of the above advice is summarised in this section I took from one of those official documents on handling throttling with the Graph API (Feb 2018)

Best practices to handle throttling

The following are best practices for handling throttling:

  • Reduce the number of operations per request.
  • Reduce the frequency of calls.
  • Avoid immediate retries, because all requests accrue against your usage limits.

When you implement error handling, use the HTTP error code 429 to detect throttling. The failed response includes the Retry-After field in the response header. Backing off requests using the Retry-After delay is the fastest way to recover from throttling because Microsoft Graph continues to log resource usage while a client is being throttled.

  1. Wait the number of seconds specified in the Retry-After field.

  2. Retry the request.

  3. If the request fails again with a 429 error code, you are still being throttled. Continue to use the recommended Retry-After delay and retry the request until it succeeds

This advise all makes sense so that if your code is making a lot of calls (think migrating SharePoint items or doing bulk updates) that the Graph may tell you to slow down. When I was investigating my scenario however, it just didn’t make sense that the code was generating enough traffic to worry the Graph (Office 365 service). The telemetry was telling me the code had made around 2,500 Graph calls spread over a period of 24 hours and this was also spread across more than 100 users from a number of different Office 365 tenants.

Diving deeper into the telemetry a pattern quickly emerged, the 429 errors were being returned in response to a Graph call to get a list item based on a column value. Something along these lines:

https://graph.microsoft.com/v1.0/sites/{site-id}/lists/{list-id}/items?filter=Fields/Title eq 'testitem'

This call didn’t fail all the time, if fact it only seemed to get the 429 error in less than 10% of the cases.

Having spend many hours over the past few years ‘working with’ SharePoint thresholds and query limitations on large lists and libraries, my mind started to move towards thinking that maybe the 429 error was a bit misleading and was actually failing due to the Graph API hitting SharePoint threshold limitations.

Off to prove my theory, I’ve got a library with just under 5000 items (which is the SharePoint list threshold)

MS-Graph-API-Throttling-00-Large-SharePoit-Library-Threshold-Cameron-Dwyer.png

Using the Graph API Explorer I can make a call that queries this SharePoint library for a specific item matching on the Title column value being equal to “upload.log” (a file which I know exists in the SharePoint library).

MS-Graph-API-Throttling-00a-Get-SharePoint-Item-By-Field-Value-Cameron-Dwyer.png

As expected  the item is found and a Success code 200 is returned along with the JSON payload in the response body shown above. Time to prove the theory, what if I now add 2 more files to the same document library and repeat the process?

After uploading 2 more files, the library settings now indicate that we have exceeded the list view threshold.

MS-Graph-API-Throttling-05-Large-SharePoint-Library-Exceed-Threshold-Cameron-Dwyer.png

Now executing the same query in the Graph API explorer gives us the 429 error code. Inspecting the response body we can see the additional error code of “activityLimitReached” and message of “The application or user has been throttled”

MS-Graph-API-Throttling-06-Activity-Limit-Reached-Cameron-Dwyer.png

Why was this error misleading? Neither the error code or message specifically indicate the issue being related to SharePoint thresholds. The documentation and best practice articles (linked to at the start of this article) regarding this 429 response are written on the premise that the volume and frequency of calls is responsible for the error and hence the guidance to handle the error should be to incrementally back-off and keep trying until you get success. This guidance is totally misguided in the case of hitting the underlying SharePoint threshold limitation as the call will always fail and has nothing to do with the volume or frequency of calls you are making. It will fail if it’s the only call you make all day and no matter how many times you retry, it will always fail.

 

Photos from the European SharePoint Conference 2017 (Dublin)

Newcastle Coders Group talk on Transitioning to Modern Office Add-in Development November 2017

imageI had a great time on my first visit to the Newcastle Coders Group this week. It was nice to meet some passionate developers and hear what people are up to outside of my usual Microsoft sphere.

Thanks for having me talk about modern Office add-in development. Here’s the slide deck from the evenings talk.

https://camerondwyer.files.wordpress.com/2017/11/transition-to-modern-office-addin-development-ncg-talk-nov-2017.pptx

coders-user-group-newcastle-cameron-dwyer-office-add-in-dev

Sydney SharePoint User Group – The Transition to Modern Office Add-in Development

sharepoint-user-group-community-sydney-cameron-dwyerI had the pleasure this week of speaking at the Sydney SharePoint User Group on the topic of transitioning to the modern Office Add-in development model.

We discussed:

  • The existing COM/VSTO Office Add-in development model
  • The reasons and drivers for needing a new development model
  • What the modern Office Add-in development is and how it works
  • Benefits of the modern model
  • What this transition means for Office developers
  • A look at the typical modern add-in technology stack and discussing some of the options
  • The wider Office Developer Vision (Extending Office through add-ins + accessing Office 365 data via Graph)

Thanks to those who attended and as promised here’s a link to the slide deck from the nights presentation.

Transitioning to Modern Office Add-in Development (slide deck)

sharepoint-user-group-sydney-cameron-dwyer-office-add-in-dev

%d bloggers like this: