LI Nick X.com
Blog |Follow Nick on Mastodon| About
 

Finding this solution via Google/Bing/etc was just impossible, so this post is an attempt to make this easier.

Some Important Information: If you're using SharePoint online, i.e. a cloud solution, something part of office365 or whatever it's called today, this post is NOT for you; there are plenty of python posts, libraries and packages for that, this is what you need to do if you business is still running a legacy on-prem solution and the normal things don't work.

This solution uses the old REST API, now there is some kind of relationship with accounts.accesscontrol.windows.net and I have no idea why?!

My story.

For "reasons" my $work has both online and on-prem solutions, the online one works great and that's what I normally dealt with but due to "reasons" I needed to upload to on-prem. After hours of googling, and trying different things in python, I gave up and wrote my solution in PowerShell using the PNP PowerShell Module which got me over the initial hurdle but since the rest of my project was in python it was only a tactical fix.

To solve this in python, I started initially with Wireshark and a man-in-the-middle proxy and pulling out the REST calls, get_access_token() was written that way but it was painful, it really wasn't clear to me how file uploads was working. I left it for a while, the tactical fix was still ok.

My first AI win!

A new scenario came up, I needed to get data from my DB and create a file in this legacy SharePoint, so either I was gonna have to start talking to my DB in PowerShell or solve this in python. As is the trend in ~2024~ 2025 we now ask GenAI, OF COURSE THE CODE SUGGESTED WAS WRONG AND DIDN'T WORK but that's what we expect right?

However, the code suggested did give me some clues, based on the suggestion I was able to start tweaking the code and ta-da! it works, I could upload the file! A few more questions, a bit of googling and tweaking I even managed to update the metadata. I did not get as far as listing/finding existing files and performing a download as currently I don't need to, but if I do, maybe I'll update this post.

Some Sharepoint Stuff...

On my SharePoint document library, I've created a custom column/field for the sha256 sum of the uploaded file; if you want that replace x0034_ha256 with whatever your field code is... notice the code is not the name, so create the column, then edit it and in the address bar of your browser you can see the code.

If you don't need the sha256, just comment out & remove update_file_properties()

My Python Code.

I hope this is useful to someone, good luck!

#!/usr/bin/env python
# coding=utf-8
# Python linter configuration.
# pylint: disable=W0718 # Broad Except
import logging
import requests # pip install requests

logging.basicConfig(level=logging.DEBUG)

CLIENT_ID = "replace_me"
CLIENT_SECRET = "replace_me_too"
SITE_HOST = 'sharepoint.domain.com'
SITE_NAME = 'ABC123'
SITE_URL = f'https://{SITE_HOST}/sites/{SITE_NAME}'
UPLOAD_FOLDER = "Documents/Uploaded Files"
SHAREPOINT_LIBRARY = "Documents"
FILE_PATH = "test.txt"
FILE_TITLE = "this is a title for test.txt"
FILE_HASH_COL = '_x0034_ha256'
FILE_HASH_VALUE = 'abc123' # Replace with the actual hash value

def get_access_token():
    """
        # Step 1: Obtain an access token
    """
    r = requests.get(f"{SITE_URL}/_vti_bin/client.svc", timeout=10, headers={'Authorization': 'Bearer', 'Accept': 'application/json'})

    logging.debug(r.status_code) # <- 401
    logging.debug(r.headers['WWW-Authenticate'])

    auth_headers = {}
    for ah in str(r.headers['WWW-Authenticate']).split(','):
        key = str(ah.split('=')[0]).strip()
        try:
            val = str(ah.split('=')[1]).replace('"', "")
        except IndexError:
            val = None

        auth_headers[key] = val

    logging.debug(auth_headers)
    logging.debug(r.json())

    data = {
        'grant_type':'client_credentials',
        'client_id':f"{CLIENT_ID}@{auth_headers['Bearer realm']}",
        'client_secret':CLIENT_SECRET,
        'scope':f"{auth_headers['client_id']}/{SITE_HOST}@{auth_headers['Bearer realm']}",
        'resource':f"{auth_headers['client_id']}/{SITE_HOST}@{auth_headers['Bearer realm']}",
    }
    response = requests.post(f"https://accounts.accesscontrol.windows.net/{auth_headers['Bearer realm']}/tokens/OAuth/2", timeout=10, data=data)
    logging.debug(response.status_code) # <- 200
    logging.debug(response.json())

    if response.status_code == 200:
        return response.json().get('access_token')

    response.raise_for_status()
    return None


def upload_file(access_token):
    """
        # Step 2: Upload the file to SharePoint
    """
    upload_url = f"{SITE_URL}/_api/web/GetFolderByServerRelativeUrl('{UPLOAD_FOLDER}')/Files/add(url='{FILE_PATH}', overwrite=true)"

    with open(FILE_PATH, 'rb') as file:
        headers = {
            "Authorization": f"Bearer {access_token}",
            "Accept": "application/json;odata=verbose",
            "Content-Type": "application/octet-stream"
        }
        response = requests.post(upload_url, headers=headers, data=file, timeout=10)

    logging.debug(response.json())

    if response.status_code == 200:
        logging.info("File uploaded successfully!")
        return response.json()  # Return the response to use the returned file info

    response.raise_for_status()
    return None

def update_file_properties(access_token, file_info):
    """
    # Step 3: Update the file properties (Title and Custom Column)
    """
    update_url = file_info['d']['ListItemAllFields']['__deferred']['uri']
    logging.debug(update_url)
    file_type = file_info['d']['__metadata']['type']
    logging.debug(file_type)
    e_tag = file_info['d']['ETag']
    logging.debug(e_tag)

    # Payload for the update
    payload = {
        '__metadata': {'type': 'SP.Data.Internal_x0020_DocumentsItem'},
        'Title': FILE_TITLE,
        'OData__x0034_ha256': FILE_HASH_VALUE
    }

    headers = {
        "Authorization": f"Bearer {access_token}",
        "Accept": "application/json;odata=verbose",
        "Content-Type": "application/json;odata=verbose",
        "IF-MATCH": "*",            # Update the file regardless of the current version
        "X-HTTP-Method": "MERGE"    # Specify the method for updating
    }

    response = requests.post(update_url, headers=headers, json=payload, timeout=10)

    if response.status_code == 204:
        logging.info("File properties updated successfully!")
        logging.debug(response.text)
    else:
        response.raise_for_status()

if __name__ == "__main__":
    try:
        token = get_access_token()               # Obtain access token
        json_info = upload_file(token)           # Upload the file with the token
        update_file_properties(token, json_info) # Update the file properties
    except Exception as e:
        logging.error("An error occurred: %s", e)

References

ha-ha-ha, GenAI doesn't do that... but this is relevant: https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/working-with-folders-and-files-with-rest

 

 
Nick Bettison ©