( 'SOFTWARE ARCHITECTURE', 24 ), # I like using to draw application infrastructure ( 'KOTLIN', 23 ), # I think this is a framework for Java/Android? ( 'TENSORFLOW', 22 ), # Google DL/ML library that I'll try to use later in this article ( 'DISTRIBUTED SYSTEMS', 22 ), # Using AWS, I guess I have technically designed distributed systems but I wouldn't call it one of my skills ( 'PHP', 22 ), # I almost learned it to support a WordPress site but opted to use JAMStack instead ( 'DATA WAREHOUSING', 22 ), # I have used Google BigQuery before which I think counts for this skill ( 'DEEP LEARNING', 20 ), # I'm going to try to use this later in this article ( 'DATA MODELING', 20 ), # To me this means writing Django models, or thinking about how to structure an API/json data, etc. I have many abandoned "rebase-practice" repos ( 'ANGULAR', 29 ), # I tried it once for about an hour, I know that people love to hate it, I'm just not sure why ( 'SWIFT', 29 ), # This is apparently a very popular language and has use cases outside of mobile development, but I've never used it ( 'LINUX', 28 ), # I spend lot of time using Linux machines, mostly Ubuntu.
#Putting a webscraper on raspberry pi mac#
I love the admin, ORM and DRF ( 'MACHINE LEARNING', 44 ), # I am familiar with some ML techniques but not very well practiced ( 'MONGODB', 43 ), # Have used it before, but I try to use the postgres JSONField for storing NoSQL data when possible ( 'IOS', 38 ), # I have an iPhone, but haven't used a Mac in a long time, mostly on Linux and Windows ( 'MYSQL', 36 ), # I tend to use Postgres, I don't think I've ever used this ( 'ANDROID', 35 ), # Not something I have worked with ( 'DATA ANALYTICS', 32 ), # I do a lot of this ( 'GIT', 30 ), # I'm very slowly trying to learn advanced git features. I prefer REST but I get the appeal (for frontend developers) ( 'GOOGLE CLOUD', 46 ), # I'm not a big GCP as I mostly use AWS and Digital Ocean but I would like to try Cloud Run ( 'RUBY', 44 ), # I haven't used it ( 'DJANGO', 44 ), # Django is my go-to tool for building web apps and APIs. I prefer ECS or Swarm, but I might try using it more ( 'GO', 58 ), # I haven't ever used Go, but it doesn't look too bad coming from Python ( 'REACT NATIVE', 58 ), ( 'C++', 55 ), # Also haven't used this, but I have a book on it ( 'GRAPHQL', 48 ), # I tried GraphQL and built a HN clone in Django. This site uses Tailwind ( 'HTML', 71 ), ( 'DOCKER', 66 ), # I am a big container fan! It is my preferred way to run software, locally and in the cloud ( 'KUBERNETES', 58 ), # I read the Manning book on k8s. Recently I'm into Tailwind and Material UI. I generally use Python for web apps ( 'POSTGRESQL', 132 ), # I have used Postgres ever since I started using Django and like it a lot ( 'TYPESCRIPT', 114 ), # this is another goal of mine for 2021, it seems like an inevitability ( 'JAVA', 79 ), # I have never used Java ( 'SQL', 74 ), # I usually don't write my own SQL queries I view SQL through the lense of an ORM ( 'RUBY ON RAILS', 72 ), # I haven't used RoR ( 'CSS', 71 ), # I like CSS Frameworks. I have really been enjoying using CDK to build infrastructure ( 'NODE.JS', 195 ), # I would to do more with node this year. [ ( 'JAVASCRIPT', 330 ), # I have been using JS a lot recently with Vue ( 'REACT', 323 ), # As a prefer to use Vue, I haven't used React in a while ( 'PYTHON', 312 ), # I'm a big Python fan, it was the first language I touched, happy to see it near the top! ( 'AMAZON WEB SERVICES (AWS)', 200 ), # I like AWS a lot. The resulting JSON structure for the big array of companies looks like this: These were all pretty easy to account for, it just required some additional logic to handle default values for s that were not included as well as mixed data types and representations where there were inconsistencies. Years of experience required was also inconsistent with mixed types like 3+ Years, Any (recent grad ok) and Senior or Juniors, for example. Some salary ranges had typos like $90k - $10k Job details (visa requirements, salary, equity) were all labelled with the same class and they were inconsistent (sometimes salary or equity or both were excluded, for example).Įquity was mostly a range of percentages such as 1% - 2% and sometimes a single percentage like 1.5%. To scrape the data I used my go-to library for this type of task: BeautifulSoup. append (company_details ) with open ( "output.json", "wb" ) as f :
find_all ( "company" ) # company stats, founders and jobsĬompany_details = extract_company_details (company )Ĭompanies.
0 kommentar(er)
