Data Thing Part 6: What am I talking about? Turns out I don't know*
Sooo the last time I wrote one of these I said I was going to hold a meeting with my team to run through a load of stuff and it was definitely going to make me get everything all organised and coherent.
Spoiler alert: it did not
We ran through how we could design our data setup in Fabric, what I'd looked at so far and what the next steps would be:
- Getting the firewall / proxy rules in place to test out an On Premise Data Gateway
- Me creating a demo workspace for my team to play about with and giving them some specific exercises from the Microsoft training to run through. Their tech knowledge is great so this is more about picking out tasks that give them a good feel for how we can use the system rather than learning to do specific jobs.
However, when it came to describing the whole project to someone higher up the chain, with no experience of Fabric or Power BI, I realised I was pretty unprepared. And too used to speaking to technical people where I can take for granted that they already know what its for and how it works.
So my main takeaway was I need to work on my messaging before we go to board for funding. I need to be able to explain
- what it is
- why its better than what we do now
- example real world applications
really quickly before people lose interest 😴. And I can't rely on gesturing wildly at a pyspark notebook and scribbling a load of circles and lines on a piece of paper.
Peace and quiet and A3 paper ✏
I often find the best way for me to focus my ideas from the blurry mess is to write out what I'm doing with real pen and paper.
I like to use A3 sheets nicked out the printer draw because it's easy to jumble them up (unlike a notepad that has to be in order) and they are BIG and don't have lines.
I now have 3 sheets of diagrams and notes and a clear answer that I wish I'd got prepared before my meeting 😅
Firewall business
For anyone else looking into setting up an On Premise Data Gateway to link to on prem databases here is the article I used to find out the security changes we needed to make: On-premises data gateway considerations for data destinations in Dataflow Gen2.
It took me a while to figure out as once I set up the gateway I could see my on prem data in the Power Query preview so it appeared to be all good. But when I ran the Dataflow Gen2 job it would fail with the error:
Query_WriteToDataDestination: Mashup Exception Data Source Error Couldn't refresh the entity because of an issue with the mashup document MashupException.Error: Microsoft SQL: A connection was successfully established with the server, but then an error occurred during the pre-login handshake.
Which was because the activator couldn't access the SQL endpoint for the Lakehouse.
* Clarification - I do know, but I didn't get all the right words assembled to explain it 🥴
Other posts with tag "data thing":