What is the minimum salary for a position like this? $150+?
Probably a long shot as this is a damn senior technology position in a newish skillset... but my team is looking at add more folks for our consulting professional services practice. This is a huge data warehouse vendor which is making a push with Hadoop appliance and services offerings... great customers with lots of different data sets.
So have you worked extensively with Hadoop? Can you intelligently explain concepts such as RCfile and talk about splittable compression in a map reduce job? Have you worked with HBase, Hive,Oozie, etc? Does standing up a cluster sound like a normal monday? Are you experienced with Java, SQL, python, shell scripting as well? Do you have excellent presentation skills and no problem working directly with customers of all levels?
If you have, let me know and I will refer you (and likely end up being one of your primary interviewers if you make it past HR).
This is a 50-100% travel job, can live anywhere and work from home.
Last edited by CnCGOD; 06-21-2013 at 09:10 PM.
What is the minimum salary for a position like this? $150+?
I can't speak for his company, but as a person who this position would be targeted towards, I probably wouldn't accept less than $180~, and only cause of the work from home aspect, if I was required to move to an expensive city that would change the valuation a lot, clearly.
Sorry for hijacking the thread but I've always had a few questions about consulting.
#1) Do you get some form of company matched retirement account?
#2) Do you get some form of company healthcare?
I could probably find a consultant job paying decently more per year but my company has an outstanding retirement benefit and health/dental/eyecare programs. It almost feels like I'd need to, at least, double my salary to match all that stuff if consultant firms don't generally off it.
Healthcare is pretty standard big company plan which for me alone the high benefit is like 2500$ out of pocket max, 500 deductible for 110$ a month or so.
401k is I put 6% in and get 4 or 5% matched.
I know the big consulting companies are similar as I have talked to them myself in the past with varying matches (some even have pensions). These are real high end consulting companies though, not the body shop small ones the field is littered with.
Last edited by CnCGOD; 06-24-2013 at 07:12 PM.
Ya big vendors and the big pure consulting names (Accenture,Deloitte,etc) all have great benefits. A lot of "consultants" in the industry are really w2 contractors, which is probably what you have seen.
Man, why would people want to work for such large companies. I'm trying to get out of one right now myself... have a couple of good offers. A paycut is worth not working at a massive company.
I think it depends a lot on the company obviously, what is your beef? Bureaucracy?
Yeah the red tape can get pretty annoying at a big company. But not being in charge of every aspect of the operation and not being requested to work 60+ hr/wk is worth it for me.
It'd be better if the trade-offs were justifiable, but most the time, it's not some well educated PM or CTO making a good call in order to deliver a better product, or to meet a critical deadline. Most the time it's just senseless bureaucracy and meaningless spec-sheets drawn up by some guy trying to justify his existence by producing "something"
I'm only slightly bitter!
Last edited by prescient63; 06-25-2013 at 01:00 PM.
One of the better aspects of Big Data is the results (analytics) are seen as giving direct business/revenue value... so even the "IT is a cost" places will give it top billing as a money maker.
If there is one thing pointy hairs like it is numbers
Big businesses usually have better career paths, if you don't like acting like a mercenary and job hopping just to get raises. I'm at a company now that's probably nearing 500 employees and I'm having trouble seeing how I will get promotions to get those sizable raises I know they won't give otherwise.
I've worked at two of the biggest consulting firms around, one in IT, other in mgmt. I'm now independent and will never work for another person again.
Career path? Hahaha. The biggest joke at the IT firm was that to get a promotion, you had to leave and come back.
If you are average- you won't get promotions. If you like your work and put an above average effort into it- Promotions AHOY.
Switching jobs is the quickest and most reliable way to get any sort of a monetary raise. I just got promoted and it was only from 120->131. Was hoping for more
You'll get a promotion when you threaten to leave.
I've been promoted twice in five years at my company
I have found that one of the big things is that most places you have to ask for it to get promoted. Those who don't ask never get promoted.
I'm kind of hijacking this thread but CnC do you have any experience with Lucene? Setting it up, indexing your data, using the API? There is a decent sized chance I'm going to join the team for writing our next generation search engine anyways using MapReduce and it appears Lucene is the Apache solution. Are there any other solutions you'd recommend we explore?
I have worked with Lucene for keyword indexing of medical documents, it is the goto choice in a lot of deployments. Have you looked into Solr? It has done a lot of the work for you in building the search capabilities on top of the Lucene library.
I saw Solr pop up a bunch when I was doing my research into Lucene last night but wasn't 100% sure the function of the product. I haven't joined the team and was just trying to wet my feet to figure out how easy/difficult it is to use MapReduce searches. I'd be on the team investigating solutions so Lucene is just one of the options to explore but it seems to be pretty well regarded as the de facto search for MapReduce.
And I'm not sure your knowledge level with Solr but what we really need to support is named indexing. Not just full text keyword searching. In our cause we have data such as:
Bassically the 001, 002 and 003 all have different meanings and conform to the MARC standards set forth by the Library of Congress. So for our "se" index this is for subject which would need to index against the 003. The 001 would be the title index and 002 being the author index. I haven't seen if there is a way of setting something like that up using Solr/Lucene.Code:<Record> <001>Spot's adventure</001> <002>Dr. Tenks</002> <003>Dog</003> </Record>
You can see examples of faceting on sites like Newegg where you can limit your search for motherboards to specific subsets of features etc
Yeah it isn't really faceting that I'm needing. We have far too many indicies for me to see how we could display facets in a proper UX manner. I am completely ignorant about Solr so I'm just trying to gather some information about the product. I don't even know exactly what a query string looks like you call to it via HTTP or anything. I assume I'd have to at least put a layer in the middle to parse my index names into something usable by Solr.
There are some built in query parsers/query syntaxes built into Solr you can choose from (or roll your own), this is the default one and I'm not sure if it helps. http://wiki.apache.org/solr/SolrQuerySyntax
I'll try and be a bit more through with the data description then.
Basically our HBase is keyed off an incremental number as the key. 1, 2, 3 or whatever. I don't know exactly how many keys there are but it is around 100M. Under these keys there are various qualifiers but the big important ones are the XML blob and holdings. Holdings basically means that a library has this book. There are over 1b of these type of entries. The XML blob is a big metadata about the book. So it has things like title, langauges, ISBN/ISSN, author, etc etc. In general here are some common searches:
"no:123456" --> Returns 123456
"li:ABC" --> Returns all numbers library ABC has holdings on
"se:cats" --> Returns all numbers that the /root/040/sa/d contains the phrase "cats" (for this argument lets say the LoC MARC standard for subjects is in the 040 subfield "a")
There are a bunch of indexes but these are just a few examples. For the most part most of the indexes are similar to the "se:" one where we simply index a field/subfield. I was just wondering if Solr could handle all these use cases.
Yup, and it sounds like it could be a single index. Which from the look of it would greatly simplify your implementation. Take what I say with a grain of salt though, this is all knowledge that I've gained via seeing how Solr would give us a unified search and talking with various companies that do support contracts for Solr. I don't have any first hand experience, but we provide similar searching to what you're describing in our product and the contractors have all given positive feedback that what I described would scale well.
Re-derailing a bit, but on the consulting topic:
There is a LOT to be said for being the money maker. If you are an IT or dev at a company where you aren't central to their primary product, you are treated completely differently than if you are the moneymaker. Its pretty easy to justify high pay when you can point at $350k in billable hours in the past year. You also (in my experience) get to do a lot of interesting and cool shit because there are generally 2 scenarios in which people are willing to pay big bucks for a team of consultants:
1) Shit is totally fucked, and they need to be rescued.
2) They want to do really cool things but have no fucking clue how to do it and don't trust their own dev team to do it (from what I've seen, this is usually a good choice too).
fixing #1 can also involve #2, so that scenario isn't all bad.
The upsides of consulting:
1) Fast salary growth
2) do cool shit
3) work with some really experienced people and able to tap those resources. I was working with a beta SDK from microsoft trying to integrate something into a windows 8 app that wasn't really supposed to be done yet. One of the main resources for that product in the country was an IM away and a guy who worked on silverlight and then windows 8 for the past 5 years was across the room. It would have taken significantly longer without their help.
4) generally a lot better culture (at our company anyway) than traditional stodgy large companies. Not startup level (although we do have a kegerator) but a long ways from office space. When your business is technology, you tend to get more smart people and fewer pointy haired bosses.
1) Can be a lot of travel, depending on the company. Might have to work at the client site even if they are local.
2) You never "own" a product that you then tinker with for years. You go in, build something (hopefully cool), and then hand it off to the customer.
3) Not a lot of fuckoff time where you can be lazy and feel like you've "earned it" from previous accomplishments. Those accomplishments were a different team and a different time, very much a "what have you done for me lately" feel to it.
There are currently 1 users browsing this thread. (0 members and 1 guests)