Python Deployment

Overview

Ververica Cloud allows you to run Python Deployments. This topic describes how to quickly set up a JAR Deployment, run it, and view the outputs on the Ververica Cloud environment.

Prerequisites

• Sign up for Ververica Cloud and set up your password after receiving the confirmation email.

Prepare the Python and input data for testing.

• Download word_count_streaming.py

• Download Shakespeare.txt

Procedure

Step 1: Create a Workspace on Ververica Cloud

1. On the Dashboard page, click New Workspace.

2. On the New Workspace page, configure the Region and Workspace Name.

Parameter	Description	Example Value
Region	We support the following regions in the VVC Beta release: Europe (Frankfurt), US West (N. California), and US East (N. Virginia).	US West (N. California)
Workspace Name	maximum 128 characters allowed characters: upper letters, lower letters, numbers, special chars: SPACE(“ “), UNDERSCORE(“_“), DASH(“-“)	yes
Resource Quota	In the Beta version, each user will have 10 CUs by default. Read-only field.	10 CUs

3. Click Create Workspace and you will be redirected to your Dashboard.

Note

After you complete the above steps, the new workspace will be in creating status. In most cases, the workspace can be created within 5 minutes.

Step 2: Upload the Python File

1. In the left-side navigation pane of the Console, click Artifacts > Upload Artifact.

Step 3: Create a Python Deployment

1. In the left-side navigation pane of the Console, click Deployments > Create Deployments.

2. Fill in the Create Deployment form with below information.

Parameter	Description	Example Value
Deployment Type	The type of the Deployment. Currently, only JAR and Python are supported.	PYTHON
Deployment Name	The name of the Deployment. Note: The name must be unique in the current Workspace.	flink-streaming-test-jar
Engine Version	In Beta release, the default engine version is vvr-0.5.0-flink-1.15	vvr-0.5.0-flink-1.15
Python URI	The URI path to the Python file to deploy.	Select the “word_count_streaming.py” from the list or upload it by clicking the icon on the right.
Entry Module	The entry point class of the program. (1) If the file that you upload is a .py file, you do not need to configure this parameter. (2) If the file that you upload is a .zip file, you must configure this parameter. For example, you can set the Entry Module parameter to word_count.	N/A
Entry Point Main Arguments	Specify the input file path. Additional dependencies will be linked to Flink’s usrlib directory (/flink/usrlib).	--input /flink/usrlib/Shakespeare.txt
Python Libraries	A third-party Python package. The third-party Python package that you uploaded is added to PYTHONPATH of the Python worker process. This way, the package can be directly accessed in Python user-defined functions (UDFs). For more information about how to use third-party Python packages, see Use a third-party Python package.	N/A
Python Archives	Archived files are currently only supported in ZIP format, such as .zip, .jar, .whl, and .egg, etc. Archived files will be extracted to the working directory of the Python worker process. For example, if the name of the compressed package containing the archived file is mydata.zip, the following code can be written in a Python custom function to access the mydata.zip archived file: def map(): with open("mydata.zip/mydata/data.txt") as f: ...	N/A
Additional Dependencies	The additional dependency file(s) for this deployment.	Select the Shakespeare.txt from the list or upload it by clicking the icon on the right.
Deploy to Session Cluster (not recommended)	This option is for deploying on the Session Cluster, which is usually used for testing purposes.	Unchecked
Description	Description of the deployment.	N/A

Step 4: Start the deployment and view the result

1. To the right of the target job name, click Start in the Actions column.

2. Fill in the startup option information.

The start options for streaming jobs are as follows:

• Initial Mode: If you have a new online job, you can start with Initial Mode. In addition, if the State cannot be reused, you can also start the job without any initial states. After you select Initial Mode, you can select Specify source table start time and provide specific time information.

• Resume Mode: You can start with an existing state.

3. Click Start.

After the deployment is started, the deployment status changes to RUNNING. This indicates that the deployment is running properly.

Step 5: View the results

1. Click the name of the desired deployment.

2. Click the Logs tab >> Running Logs tab >> choose Running Task Managers

3. On the Running Task Managers tab, click the value in the Path, ID column.

4. Under the Log List tab, click the flink*.out file and you will see the results.