In my not-so-humble opinion, putting your client-server app on an AWS or Azure VM is no more "Cloud" solution than running DynamoDB and telling people you've got "Big Data".
To me, real Cloud and real Big Data start with using scalable PaaS OS, like Azure or AWS. If you got no PaaS, don't talk cloud and big data.
Tuesday, April 28, 2015
Learning Azure on Your Own Dime - Extremely Expensive
OK, I am completely sold on The Cloud, and specifically on PaaS. I am ready to go use it, especially Azure since I like Microsoft stack and tools, despite AWS being more popular and having more features (we, artisans ;-) develop stronger connection with our tools and techniques, than craftsmen or business people.) I started learning Azure, got the account, a VM, Azure SDK, developed an interesting testing-the-waters project - the Azure Storage Queue multicaster, similar to ESB topics.
My first Azure bill, despite MSDN subscription was above $150. I run a 24/7 application server on a slow VM there, where majority of the cost comes from, but small experiments, like with the VPN for example, added $25 before I noticed that it's really expensive. That was unpleasant.
Now, I want to experiment with something other than what I can run on local Azure simulator (Tables, Blobs and Queues). I would like to play with ESB, API Management, DocumentDB and just about everything else, but I am really concerned about how much cost I am going to incur by using each part of the stack, especially if I forget to power everything down on Azure for a night. The point is, my impression is that simple poking around and learning Azure stuff is rife with unpleasant and costly surprises. All these risks and impediments combine into a single big turn off for a potential Azure developer.
With Azure lagging in adoption behind AWS, slapping down even dedicated fans like myself, means that Microsoft is very unlikely to ever catch up with AWS. "Developers! Developers! Developers!" indeed. I know it is possible to plan the cost ahead using complex Excel spreadsheet, and there is an almost-meaningless 30-day free trial (learn new enterprise OS in 30 days, really??), but all these impediments add up to one huge fogget-bout-it!
Here's what Microsoft could do to make it easy for me to learn Azure: create closed-off sand-box on Azure infrastructure, inaccessible from the outside, but make all Azure stack technologies and components available there for free to all MSDN subscribers. Let us play with and learn Azure for free, and charge us only when our wares is made available public. That would be a huge differentiator for Azure.
If you agree with the statement above, talk at Microsoft about this.
But while Microsoft is ignoring us, I decided to do my part, in the upcoming weeks and months will do public service so to speak, and will pay out of my pocket for learning Azure, and will blog about my experience, including costs. I will tag those posts with "azure" and "paas". Follow this blog to take pleasure in my future misfortunes on the path of conquering Azure.
My first Azure bill, despite MSDN subscription was above $150. I run a 24/7 application server on a slow VM there, where majority of the cost comes from, but small experiments, like with the VPN for example, added $25 before I noticed that it's really expensive. That was unpleasant.
Now, I want to experiment with something other than what I can run on local Azure simulator (Tables, Blobs and Queues). I would like to play with ESB, API Management, DocumentDB and just about everything else, but I am really concerned about how much cost I am going to incur by using each part of the stack, especially if I forget to power everything down on Azure for a night. The point is, my impression is that simple poking around and learning Azure stuff is rife with unpleasant and costly surprises. All these risks and impediments combine into a single big turn off for a potential Azure developer.
With Azure lagging in adoption behind AWS, slapping down even dedicated fans like myself, means that Microsoft is very unlikely to ever catch up with AWS. "Developers! Developers! Developers!" indeed. I know it is possible to plan the cost ahead using complex Excel spreadsheet, and there is an almost-meaningless 30-day free trial (learn new enterprise OS in 30 days, really??), but all these impediments add up to one huge fogget-bout-it!
Here's what Microsoft could do to make it easy for me to learn Azure: create closed-off sand-box on Azure infrastructure, inaccessible from the outside, but make all Azure stack technologies and components available there for free to all MSDN subscribers. Let us play with and learn Azure for free, and charge us only when our wares is made available public. That would be a huge differentiator for Azure.
If you agree with the statement above, talk at Microsoft about this.
But while Microsoft is ignoring us, I decided to do my part, in the upcoming weeks and months will do public service so to speak, and will pay out of my pocket for learning Azure, and will blog about my experience, including costs. I will tag those posts with "azure" and "paas". Follow this blog to take pleasure in my future misfortunes on the path of conquering Azure.
Monday, April 27, 2015
Photography is Easy
The truth is, good photography, although not exactly trivial, still requires a lot less skills than, say, painting, or playing a musical instrument. Anyone with a modest doze of persistence can figure out basic relationships of ISO, aperture and shutter speed, effects of different focal length, white balance and tone curve. Then all it takes to make a few good pictures is to take thousands of whatever pictures. Here is the result of my cherry-picking a few shots I mined from my digital collection. Most of them shot on the modest Canon PowerShot S95
and lightly edited in Picasa.
On a PC, hit F11 to put your browser to full-screen and then click on the picture to see it in the album.
On a PC, hit F11 to put your browser to full-screen and then click on the picture to see it in the album.
 |
| Hakone, Japan |
 |
| Rain in Wyoming |
 |
 |
| Paris, of course |
 |
| Seattle Skyline |
 |
| Savannah |
 |
| Caminito, Buenos Aires |
 |
| Yellowstone |
 |
| Key Largo, FL |
 |
| Chicago |
 |
| Ukraine |
 |
| Kauai, HI |
 |
| Napali Coast |
now?
A Little Help?
If you are planning to shop on Amazon, please use this Amazon link to get there - it will help me with, um.. being inspirated™ to maintaining this blog.
Thanks!
Thanks!
Sunday, April 26, 2015
But how long will it REALLY take?
Most programmers, myself included, are notoriously bad at estimating how long it will take to finish the job. So after much research I am ready to finally make the estimates very precise and scientific: whatever number programmer gives you, use The Vlad Law and multiply the estimate by PI. Not by 3 - that would be just a worthless guess. No, multiply it by 3.1415 - that's the objective truth, I tell ya!
What Is Good Software Architecture?
Ask any software architect a question what makes good design, and you will get a good number of very valid criteria. The one I don't hear mentioned, and one that is critical for me when I design software, is how quickly a mid-level engineer can start and keep following processes, templates and practices I create as part of the design. If design is so sophisticated that it requires superstar engineers to deal with that, that's not a good design no matter how advanced are concepts and frameworks built into the design. It's already too expensive.
Saturday, April 25, 2015
Everyone, To The Cloud!
A few years ago I joined a company that was in the process of selecting new ERP system. Vendors were usual suspects: consultants peddling SAP, Microsoft, Oracle and some other ERPs. I got to quiz vendors regarding different features and capabilities of the systems they were selling. I wanted to find how how costly integration with my company other enterprise systems would be, and asked every vendor whether their system exposed their functionality as web services. All of them did, of course. Then I asked them what I considered a very logical and simple next question. I told them, since we are going to orchestrate data updates across multiple disparate systems, I needed to know whether their web services supported transaction, like WS-Trasaction from the WS-* stack? I got blank stares and promises to find that out. I asked whether they were ever asked about this before by other enterprise architects, and to my huge surprise all of them said no. I asked how many ERP implementations they had under their belts, and all of them had dozens. Later all vendors got back to me with an astonishing information: none of their systems' web services supported transactions. That meant that garbage data were bound to accumulate over time and nobody even thought that was a problem.
That struck me as very odd and made me think: in my day-to-day life even mid-level developers usually have decent grasp of what transactions are for and often don't need supervision in applying them, as long as we are talking about SQL programming or writing data access layer of business applications. But as soon as people leave database world, somehow even professional enterprise software integrators become completely unconcerned about transactions. That matched my experience of virtually every enterprise system I ever encountered having lots of garbage data in it, requiring lots of effort/money to cleanse data. But the conclusion was inescapable: by and large, as a practical matter, corporations are pretty comfortable with not having transactions guarding integrity of their data. As a matter of fact, companies don't care about their data consistency.
Now, my thinking went like this: if people voluntarily give up transactions without getting anything in return, what can be gained if transactions are avoided not by neglect, but instead, by design? Well, if we believe CAP theorem, letting go of data consistency should let us gain high availability and partition tolerance, which translates into high scalability. And, ladies and gentlemen, that's what "big data" management systems offer: high scalability and high availability if you can deal with eventual consistency of data. And since lots of companies are not even trying to achieve data consistency, switching to NoSQL-based "big data" platforms becomes a no-brainer.
Now, NoSQL and "big data" have becomes such an incredibly abused buzzwords, that I need to stop for a moment and state that, for example, MongoDB, in my opinion, although a very fast NoSQL data management system, is not necessarily a *bit data* system, because it was not designed to be one - it was built for speed and sacrificed parts of CAP to achieve high performance. Then let's look at Hadoop. No question that's a big data management system. But the main problem is that it's not for on-line data processing - it's strictly batch processing using map/reduce approach. And if you want to set up Hadoop cluster, it's a pretty expensive proposition.
All that said, I argue that first truly useful general-purpose big data on-line processing data management system was Amazon AWS DynamoDB. It has eventual consistency, no transactions, limited returned data set size, and other limitations, but it scales in a nearly linear manner. Then Microsoft came up with Azure Tables and now Document Database. Even though you may say that eventual consistency in not really what data online processing is, I say these systems latency is tolerable enough that these systems could be considered pseudo-online.
Now lets review the landscape again: transactions are abandoned, non-transactional highly-scalable data management systems are available as a part of PaaS stacks from Amazon and Microsoft, so... there is pretty much no reason to have your data processing strategy depend completely on ACID databases. Moreover, if we, developers, train ourselves to deal with more complex DAL tiers underpinned by Azure and AWS eventually-consistent big data engines, there is no real reason to use ACID databases as a default position, which is equal to "everyone, to the cloud!"
That struck me as very odd and made me think: in my day-to-day life even mid-level developers usually have decent grasp of what transactions are for and often don't need supervision in applying them, as long as we are talking about SQL programming or writing data access layer of business applications. But as soon as people leave database world, somehow even professional enterprise software integrators become completely unconcerned about transactions. That matched my experience of virtually every enterprise system I ever encountered having lots of garbage data in it, requiring lots of effort/money to cleanse data. But the conclusion was inescapable: by and large, as a practical matter, corporations are pretty comfortable with not having transactions guarding integrity of their data. As a matter of fact, companies don't care about their data consistency.
Now, my thinking went like this: if people voluntarily give up transactions without getting anything in return, what can be gained if transactions are avoided not by neglect, but instead, by design? Well, if we believe CAP theorem, letting go of data consistency should let us gain high availability and partition tolerance, which translates into high scalability. And, ladies and gentlemen, that's what "big data" management systems offer: high scalability and high availability if you can deal with eventual consistency of data. And since lots of companies are not even trying to achieve data consistency, switching to NoSQL-based "big data" platforms becomes a no-brainer.
Now, NoSQL and "big data" have becomes such an incredibly abused buzzwords, that I need to stop for a moment and state that, for example, MongoDB, in my opinion, although a very fast NoSQL data management system, is not necessarily a *bit data* system, because it was not designed to be one - it was built for speed and sacrificed parts of CAP to achieve high performance. Then let's look at Hadoop. No question that's a big data management system. But the main problem is that it's not for on-line data processing - it's strictly batch processing using map/reduce approach. And if you want to set up Hadoop cluster, it's a pretty expensive proposition.
All that said, I argue that first truly useful general-purpose big data on-line processing data management system was Amazon AWS DynamoDB. It has eventual consistency, no transactions, limited returned data set size, and other limitations, but it scales in a nearly linear manner. Then Microsoft came up with Azure Tables and now Document Database. Even though you may say that eventual consistency in not really what data online processing is, I say these systems latency is tolerable enough that these systems could be considered pseudo-online.
Now lets review the landscape again: transactions are abandoned, non-transactional highly-scalable data management systems are available as a part of PaaS stacks from Amazon and Microsoft, so... there is pretty much no reason to have your data processing strategy depend completely on ACID databases. Moreover, if we, developers, train ourselves to deal with more complex DAL tiers underpinned by Azure and AWS eventually-consistent big data engines, there is no real reason to use ACID databases as a default position, which is equal to "everyone, to the cloud!"
A Few Pictures From Piedmont Park, Atlanta
Here are the few shots I made recently with my new Canon 6D with EF24-105mm IS lens
. I shot RAW and did a relatively quick post work with Adobe Lightroom 5
.
_
. I shot RAW and did a relatively quick post work with Adobe Lightroom 5
.
Subscribe to:
Posts (Atom)