Write us what you want & improve the DevOpsCloud website for easy to use.
To stop spammers/bots in Telegram, we have added a captcha while joining the Telegram group, which means every new member, has to authenticate within 60 seconds while joining the group.
Home >>All Articles
All Articles
Published Articles (117)
AVR posted:
3 years ago
Why do we need Delta Lake?
Challenges in implementation of a data lake
Missing ACID properties (A.C.I.D. properties: Atomicity, Consistency, Isolation, and Durability)
Lack of Schema enforcement
Lack of Consistency
Lack of Data Quality
Too many small files
Corrupted data due to frequent job failures in PROD (We cannot rollback)
Challenges in implementation of a data lake
Missing ACID properties (A.C.I.D. properties: Atomicity, Consistency, Isolation, and Durability)
Lack of Schema enforcement
Lack of Consistency
Lack of Data Quality
Too many small files
Corrupted data due to frequent job failures in PROD (We cannot rollback)
View replies (0)
Posted in: Azure | ID: Q73 |
September 20, 2021, 06:34 AM | 0 Replies
September 20, 2021, 06:34 AM | 0 Replies
Recent Articles (117)
- Q118. What is Cloud? Cloud is nothing, but we can get the Infrastructure for our project easily. When we use Cloud, we don't have to invest money into servers/...
- Q117. Python contains the following data types. Int Float Complex Bool Str Bytes Bytearray Range List Tuple Set Frozenset Dict None
- Q116. List of reserved words/keywords in python: We have 33 reserved words in python True, False, None (T, F and N are uppercase) and, or, not, is if, ...
- Q115. What is a python identifier? What is an Identifier? A name in a python program is called an Identifier. It can be a variable name, class name and method...
- Q114. What do you know about the EditPlus editor? This is the most commonly used editor for python. Go to editplus.com and download Editplus version Download ...
- Q113. What is AKS? ------------------- AKS stands for Azure Kubernetes Service We should understand the difference between Monolithic vs Microservices Monoli...
- Q112. How to write a program in notepad and save it locally? -------------------------------------------------------------------------- Write the code and sa...
- Q111. What is IDLE? Integrated Development Learning Environment It is a REPL tool responsible for reading, evaluating, printing and loop R-Read E-Evaluate ...
- Q110. Python Installation: --------------------------- Go to the official website -> www.python.org Download the latest python exe file for Windows Run the...
- Q109. Features of python: -------------------------- Simply and easy to learn Open source High-level language Platform independent Dynamically typed(Type r...
- Q108. What is python? Python is a programming language. Python is widely used in the IT Industry today. We can develop applications using a programming langua...
- Q107. can i get a help for python scripting for automating devops implementation?
- Q106. What do you know about the s3 bucket in AWS? Amazon S3 stands for Amazon Simple Storage Service. We can store files/images/videos/log files/war files/ob...
- Q105. Let's learn something about MFA(Multi-factor authentication) Go to AWS Account at header menu - click on Security Credentials This is where we can see th...
- Q104. How to delete IAM users in AWS? Go to IAM Click on users Select the appropriate users Click on the Delete button If any prompt comes, follow the AWS ...
- Q103. What is IAM & What do you know about IAM in AWS? IAM stands for Identity Access Management Let's assume that we have 100 users in the company, and all ...
- Q102. How to install Jenkins on AWS EC2 Linux Machine? Go to AWS Launch an Instance Name-Provide a name OS Image- I pick Red Hat Linux OS Instance type - t...
- Q101. What do you know about web servers and application servers? Examples of web servers are Nginx or apache. Examples of Application servers are Tomcat/Weblo...
- Q100. What is EC2? EC2 is nothing Elastic Cloud Computing It is nothing but an instance or VM. How to launch Instance? Click on the launch Instance Name i...
- Q99. SPARK ARCHITECTURE: ------------------------------------ There are 5 parts to it ->Driver program ->Cluster Manager ->Worker Node ->Executor ->Tas...
- Q98. What do you know about Spark Architecture? SPARK ARCHITECTURE: ----------------------------------- There are 5 parts to it and let's understand how th...
- Q97. Real-time data processing vs. Batch processing: ---------------------------------------------------------------- ->Real-time data collects the data and p...
- Q96. Why are we using spark when map-reduce was there in the IT Industry? -------------------------------------------------------------------------------------...
- Q95. What is the spark? Why do we need a spark? Why organizations are choosing spark to transform the data other than map-reduce/ADF? ->spark is in-memory ...
- Q94. What is logging in Python? When we use python code in Databricks to develop spark code, we need to use the python logging module to implement logging W...
- Q93. What is File Handling in Python? We manage the files Why do we need to manage the files? What is there in the files? The normal file would have data in...
- Q92. What do you know about Exception handling in python? Exception handling would be there in all programming languages like .Net java python SQL scrip...
- Q91. Let's understand the Data flow architecture in Azure databricks. We need to have good knowledge of how different components are connected and internally w...
- Q90. What is Databricks & What do you know about Databricks? Databricks is a new analytics service. Azure databricks is a fast, easy, scalable, and collabor...
- Q89. How do we create a notebook in databricks, and what are our options while creating a notebook? We can create a notebook using any of the below languages....
- Q88. What do you know about Databricks runtime(DBR)? The set of core components that run on the clusters managed by Databricks is nothing but DBR. Databrick...
- Q87. What do you know about pool in databricks? The Instance pools API allows you to create, edit, delete and list instance pools. A set of idle, ready-to-u...
- Q86. What do you know about Computation Management in Databricks? We have two types of clusters in databricks 1) Interactive cluster or all-purpose-cluster...
- Q85. What do you know about Data Management in Databricks? We can manage the data and also we can upload the data. When we log in to databricks, we can go to ...
- Q84. What is a workspace in Databricks? Databricks Workspace is an environment for accessing all of your Databricks assets. The dashboard is a ...
- Q83. Databricks Community Edition is FREE for self-learning. There are three ways to interact with Databricks Interface. UI - Using UI, we can create a cluste...
- Q82. Need to join telegram group, it always say that is private group. My telegram id is @Enigma1337 One Question how can I answer for any asked question ?
- Q81. What is VNet Peering? To establish the communication between two virtual networks in two different regions. These virtual networks can be in the same reg...
- Q80. What do you know about Network Security Group(NSG) in Azure? Group of network security rules is nothing but NSG Create a network security group with rul...
- Q79. Virtual Network Service Endpoints in Microsoft Azure Virtual Network(VNet) service endpoint provides secure and direct connectivity to Azure services ov...
- Q78. What are the three different types of services we have in Azure Cloud? Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Softwar...
- Q77. Microsoft provides Azure support options in multiple ways. Basic Developer Standard Professional Direct If you’re looking for a comprehensive, o...
- Q76. How to organise Azure resources effectively? Azure provides four levels of management scope: Management groups Subscriptions Resource groups Resou...
- Q75. What are the features of Delta Lake? An open-source storage format that brings ACID transactions to Apache Spark and big data workloads. The key featur...
- Q74. What is Delta Lake? Delta Lake is an open-source storage layer that brings ACID transactions to Apache Spark and big data workloads on Data Lakes. Delt...
- Q73. Why do we need Delta Lake? Challenges in implementation of a data lake Missing ACID properties (A.C.I.D. properties: Atomicity, Consistency, Isolation...
- Q72. Let's learn VNet peering in Azure. By default, there is communication between virtual machines in the same virtual network. We need VNet peering to e...
- Q71. Let's learn how to deploy Linux Virtual Machine using Azure Portal? Let's learn the basic understanding of the below ones, to begin with. Resource Grou...
- Q70. Let's learn how to deploy Windows Virtual Machine using Azure Portal? Let's learn the basic understanding of the below ones, to begin with. Resource Gr...
- Q69. Let's understand the networking basics to use in Azure. Network Security Group is nothing but a Group of Network Security Rules. Network Security Rul...
- Q68. We have different ways of accessing Data lake storage. A)Mount ADLS Containers to DBFS using service principal and OAuth 2.0 B)Use access keys directly ...
- Q67. How to deploy Azure Virtual Machine using CLI? CLI stands for common line interface. Deploying via CLI is very fast rather than doing via Azure Portal...
- Q66. How to delete a Saved Wi-Fi Network From the Command Prompt in Windows? Launch a Command Prompt window as Administrator. Type the following command a...
- Q65. Good resources for developing jenkins deployment files
- Q64. What is the difference between base image and Dockerfile image. Can I run Application with base image
- Q63. Let's learn how to create an Azure account. https://portal.azure.com What do we need to create an Azure account? We need an Email Id We need to solve...
- Q62. Let's have a basic understanding of Elastic Beanstalk in AWS. Elastic Beanstalk is mainly used by developers Developers write the code for creating app...
- Q61. Let's have a basic understanding of Cloud Formation in AWS. Cloud Formation: W0e create the AWS Infrastructure by writing & running the code. We ha...
- Q60. Let's learn about Cloud Trial in AWS. Cloud Trial is an auditing service. The root user can track all the history from Event history. Let's say so...
- Q59. Let's learn about CloudWatch Service in AWS. CloudWatch is a monitoring service provided by AWS. The monitoring helps you get the CloudWatch metrics ...
- Q58. Let's learn about SNS in AWS. SNS(Simple notification service): ========================= Notifications are nothing but alerts. When auto scaling l...
- Q57. Let's learn about SQS in AWS. SQS(Simple Queue Service): ====================== SQS is a message queue used to store messages while waiting for a com...
- Q56. Let's understand AWS RDS. RDS stands for Relational Database Service. RDS is used by the clients to host their databases. Relational databases ar...
- Q55. Let's understand Route 53 in AWS. Route 53 is nothing but Domain Name System. 53 is nothing but DNS Port Number Route 53 is a highly reliable and ...
- Q54. Let's learn how to create NACL(Network Access control list)? Create NACL Name - Specify the name correctly VPC - Select VPC correctly Create Now w...
- Q53. Let's learn the difference between Security group & NACL(Network Access control list) The security group will provide security at the Instance level. ...
- Q52. Let's learn how to create VPC, Subnets, Internet Gateway & Route table. How to create VPC? Name-myvpc IPv4-10.0.0.0/16 Create Assume that we have...
- Q51. Let's learn how to create a subnet in VPC? Just to recap of creating VPC. Name - myVPC IPv4 CIDR block - 10.0.0.0/16 {Technically 10.0.0.0. is a priva...
- Q50. Let's learn about subnets in AWS. What is a subnet? A subnet is a partition that is created inside the VPC We shouldn't have everything in one single...
- Q49. Let's learn about VPC in AWS. VPC stands for Virtual Private Cloud VPC is a virtual data centre in the Cloud VPC is nothing but creating a partiti...
- Q48. Let's learn how to create an IAM custom role & assign that to EC2 Instance. The role is a replacement of credentials. In simple terminology, a role w...
- Q47. Let's learn how to use AWS CLI in AWS IAM. To work with AWS CLI Access, we need Access Key ID & Secret Access Key. Go to IAM Dashboard, where we ca...
- Q46. Let's learn how to create User groups in AWS IAM. 1st of all, IAM stands for Identity and Access Management. IAM is a part of Security, Identity & Co...
- Q45. Let's learn how to create users in AWS IAM. 1st of all, IAM stands for Identity and Access Management. IAM is a part of Security, Identity & Complian...
- Q44. I want to learn devops with AWS or gcp Or azure may be
- Q43. Are you an Azure Engineer with Databricks experience? We're looking for only working professionals with Databricks hands on. If interested, please an...
- Q42. Let's learn about AWS Identity and Access Management (IAM), which securely manages AWS services and resources. IAM is related to administration. AWS ...
- Q41. Let's learn about Life cycle management which is one of the AWS S3 features. Life cycle management: ================== Go to the bucket Click on Mana...
- Q40. Let's learn about the Bucket policy, which is one of the AWS S3 features. Bucket policy: (Applicable only at Bucket level) ============================...
- Q39. Let's learn about the Access control list, which is one of the AWS S3 features. ACL(Access control list): =================== Using ACL, we can contro...
- Q38. Let's learn about Encryption which is one of the AWS S3 features. Encryption: (Data gets encrypted and saved into the bucket) ========= We use Encrypt...
- Q37. Let's learn about Transfer Acceleration which is one of the AWS S3 features. Transfer Acceleration: ============== S3 TA enables fast, easy & secure t...
- Q36. Let's learn about Cross-region replication(CRR) which is one of the AWS S3 features. Why do we need CRR? To avoid network traffic, usually, companies im...
- Q35. What do you know about Amazon S3 Storage Classes? Amazon S3 offers a range of storage classes designed for different use cases. These include S3 S...
- Q34. Let's understand how we can use static website hosting in AWS. By default, static website hosting is disabled in AWS & we need to enable this by going t...
- Q33. Let's learn about AWS S3 Versioning. AWS S3 Versioning has two advantages i)We can recover deleted objects easily ii)We can maintain different version...
- Q32. Let's learn something about AWS S3 Features: The below are the most important S3 features used by most of the companies Versioning Static website h...
- Q31. Let's understand about AWS S3 S3 stands for Simple Storage Service S3 is a storage service, and this is paid service at the Enterprise level. What typ...
- Q30. Let's understand more about volumes in AWS. Scenario1: (Same Region) Let's say that we have two EC2 Instances. 1st Instance is with Root and EBS volume...
- Q29. Should we go with AMI or Snapshots in AWS? root volumes - we go with AMI EBS volumes - we go with Snapshots
- Q28. What is AMI in AWS? AMI stands for Amazon Machine Image We can take the complete image/backup of any EC2 Instance easily. How do we create AMI? Thi...
- Q27. Let's understand more about the EC2 dashboard. EC2 Dashboard is Region specific If we create EC2 machines in one region, then they won't be visible in ...
- Q26. What is Scale-up in AWS? What is Scale down in AWS? Scale-up and Scale down also known as Vertical scaling in AWS What is Scale in AWS? What is Scale-o...
- Q25. How do we protect critical EC2 Instances without any accidental termination? To overcome the accidental termination, we have a termination protection fe...
- Q24. What is status checks in AWS? We need to understand the EC2 Instance status check when the instance is terminated. We also have to understand the EC2 I...
- Q23. What is Autoscaling in AWS? Autoscaling is nothing but an extension to load balancing in AWS. We use load balancing along with the Autoscaling featur...
- Q22. Basic Linux Commands for day to day life DIRECTORY STRUCTURE /home- users /var –log files (debugging) /etc – configuration files /tmp – t...
- Q21. What is a Load balancer & Why do we need a Load balancer? A load balancer accepts incoming traffic from users/clients and routes requests to EC2 Instanc...
- Q20. How to launch AWS Linux Instance? EC2 stands for Elastic Compute Cloud In this, let's discuss how to create Linux EC2 Instance from AWS Management Co...
- Q19. How to launch AWS Windows Instance? EC2 stands for Elastic Compute Cloud In this, let's discuss how to create Windows EC2 Instance from AWS Managemen...
- Q18. What is AWS? AWS is nothing but a Cloud Platform. In 2006, Amazon officially launched AWS(Amazon Web Services), which has become one of the major pro...
- Q16. Below NEW FREE Udemy Courses have been added on June 24, 2021, Masterclass- Kubernetes and Openshift https://www.udemy.com/course/masterclass-kubernet...
- Q15. What are the advantages of DevOps? 1. Ensure faster deployment 2. Stabilize work environment 3. Significant improvement in product quality 4. Automatio...
- Q14. What are the different types of Routing policies we have in Route 53? Simple Weighted Latency Failover Geolocation
- Q13. I am Looking For Job Oriented Project in DevOps for the Fresher
- Q12. LINUX FREE Udemy Courses The courses on Udemy, once bought, are accessible for a lifetime. They do not expire. They do not have a specific time du...
- Q11. DevOps FREE Udemy Courses The courses on Udemy, once bought, are accessible for a lifetime. They do not expire. They do not have a specific time d...
- Q10. GCP FREE Udemy Courses The courses on Udemy, once bought, are accessible for a lifetime. They do not expire. They do not have a specific time dura...
- Q9. Have you accidentally deleted Default VPC from your AWS Account? Do you want to recreate your Default VPC using AWS Management Console? It's straight...
- Q8. Azure devops course
- Q7. I want GCP course material. Can someone help me, please?
- Q6. I'm looking for AWS Material/Documents
- Q5. I want devops material
- Q4. I want Aws Solutions Architect Course
- Q3. AZURE FREE Udemy Courses The courses on Udemy, once bought, are accessible for a lifetime. They do not expire. They do not have a specific time ...
- Q2. AWS FREE Udemy Courses The courses on Udemy, once bought, are accessible for a lifetime. They do not expire. They do not have a specific time du...
- Q1. What is DevOps? DevOps is an innovative method for building and delivering quality software. DevOps is a combination of Dev+Ops where Dev stands for ...
Recent Replies (8)
- Q118R1. test
- Q82R1. t.me/joinchat/Dydn8Ct1SI5lYTZl Join using the above link and make sure that you answer the captcha question to avoid spamming. It's a simple maths qu...
- Q68R1. #secrets/createScope. This URL is case sensitive; scope in createScope must be uppercase.
- Q16R1. FREE Udemy Courses that were added on June 24, 2021.
- Q13R1. I doubt if that is possible. The best way to learn DevOps is, to begin with, FREE Udemy DevOps Courses and see how it goes https://www.devopscloud.guru/?ca...
- Q8R1. There are many FREE Udemy Azure Courses from the below link https://www.devopscloud.guru/?category=4#qstnTopp Hope it helps.
- Q4R1. Have you checked FREE Udemy AWS Courses? https://www.devopscloud.guru/?category=4#qstnTopp
- Q6R1. Please check FREE Udemy Courses where you get a lot of FREE AWS Courses. https://www.devopscloud.guru/?category=4#qstnTopp
