Why developer experience doesn't matter, as expressed by and - Dev-ex is...
Why developer experience doesn't matter, as expressed by #AWSGlue and #Oracle
- Dev-ex is expensive, only developers enjoy it
- Punching the developer in the face (metaphorically) is immensely satisfying to product managers and is free
- If you don't metaphorically punch developers in the face they will get uppity and use your product.
- They don't deserve to use your product
Self-replies
The first experience a developer should have with #spark should not be counting words. Instead, it the product should just punch the developer in the face, over and over.
`Select * From oracle table` should be so far out of reach that should a developer ever succeed, then counting words will feel like a miracle.
AWS Glue
- No VPC/Subnet for compute
- Unless you use "connnectors" (opaque java/jdbc in docker things)
- python log to info? ok. log to debug/warn? Kaboom, opaque message from java
- Where are the logs? Oh, like 10+ places to look, sometimes cloudwatch, sometimes s3
- you can install libs with pip, but not oracle stuff, that requires arm64 binaries that don't exist
- You have data frames! But they are eager evaluated and select the whole table first. :(
- 100s of databases supported! 10s of pages of documentation, or about 6 lines of documentation for your particular data source.
- What to use ray or pyspark? Well, the choice is easy, because no matter which one you choose, the experience is just pain- no VPC, no subnets, it's arm64, logs and error logs are splatter all over the place