Apache Spark is a cluster computing system that supports users to submit applications to management nodes and distribute them to the cluster for execution. If the management node does not enable ACL (access control), we will be able to execute arbitrary code in the cluster. This reproduction is built with one click using docker+vulhub: docker-compose up -d After the environment is started, four po... »
Foreword: Today I would like to share with you oneOpen sourceTake private jobs/graduate projects/part-time jobs to make moneyIt is an open source project that separates the front and back ends of Java. It took about an hour to download the source code and run it. It felt pretty good. I encountered some small problems when running the front-end code and quickly solved them according to the error pr... »
Article directory The following two tables are explainedSELECTDISTINCTWHEREAND ORINBETWEENwildcardLIkeORDER BYfunctioncity tableString functionsCommonly used function examples:concatsubstrtrimregionreplacegroup byhaving Aliassubqueryexiststable linkUse subqueries to implement multi-table queriescreate viewunion intersection valueNo intersection valuecaseThe difference between null and no value (... »
This article introduces filters in Hbase: 1. Row key filter 2. Column families and column filters 3. Value filter 4. Other filters The format is as follows: scan’table name’,{FILTER=>filter(comparison operator,’comparator’)”} 1. Row key filter RowFilter can be used with comparators and operators to compare and filter row key strings. For example, to match data grea... »
spark-parent <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.offcn.bigdata</groupId> <artifactId>spark-parent</... »
Article directory 1. Distributed ID implementation solution1.1.uuid1.2 Database primary key auto-increment1.3 Redis auto-increment1.4 Segment mode1.5 Snowflake algorithm (snowflake)1.5.1 Baidu (uid-generator)1.5.2 Meituan (Leaf) The so-called id is a mark that can be used as a unique identifier.In our daily design, for a single architecture, we generally use the auto-increment ID of the database a... »
Source code acquisition: Download from “Resources” on the blog homepage! 1. Project operationEnvironment configuration: Jdk1.8 + Tomcat8.5 + mysql + Eclispe (IntelliJ IDEA, Eclispe, MyEclispe, Sts support) Project technology: JSP + Servlert + html+ css + JavaScript + JQuery + Ajax, etc.; Personal Center Controller: /** * Personal Center Controller */ @Controller public cl... »
vi /etc/profile source /etc/profile cd /opt/hbase/conf vi hbase-env.sh export JAVA_HOME=/opt/jdk1.8 (remove #) export HBASE_CLASSPATH=/opt/hadoop/etc/hadoop export HBASE_PID_DIR=/var/hadoop/pids (remove #) export HBASE_MANAGES_ZK=TRUE (use the built-in zookeeper) (The above codes have their own locations, you have to find them, they are not typed in order) vi hbase-site.xml <configuration> ... »
Static settings on demand Reference link：Stack OverflowThe following case contains the three aspects in the title: number of executor cores, number of executors, and executor memory. Parameters such as driver memory are not mentioned in the original text. Case: 6 nodes, 16 cores + 64 GB RAM per node Since each executor is a JVM instance, we can assign multiple executors to each node. Reserve resou... »
Source code acquisition: Download from “Resources” on the blog homepage! 1. Brief description of the project Function: User’s email registration, verification code verification and user login. There is no need to register an account, and you can also upload temporary files that meet the conditions, but they are only valid for 4 hours. File management, upload, download, rename, delete, ... »
