FileTransform

Transform an ION file from Kestra's internal storage with a Jython script.

yaml
type: "io.kestra.plugin.scripts.jython.FileTransform"

Extract data from an API, add a column, and store it as a downloadable CSV file.

yaml
id: etl_api_to_csv
namespace: company.team

tasks:
  - id: download
    type: io.kestra.plugin.fs.http.Download
    uri: https://gorest.co.in/public/v2/users

  - id: ion_to_json
    type: io.kestra.plugin.serdes.json.JsonToIon
    from: "{{ outputs.download.uri }}"
    newLine: false

  - id: write_json
    type: io.kestra.plugin.serdes.json.IonToJson
    from: "{{ outputs.ion_to_json.uri }}"

  - id: add_column
    type: io.kestra.plugin.scripts.jython.FileTransform
    from: "{{ outputs.write_json.uri }}"
    script: |
      from datetime import datetime
      logger.info('row: {}', row)
      row['inserted_at'] = datetime.utcnow()

  - id: csv
    type: io.kestra.plugin.serdes.csv.IonToCsv
    from: "{{ outputs.add_column.uri }}"

Transform with file from internal storage.

yaml
id: jython_file_transform
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: file_transform
    type: io.kestra.plugin.scripts.jython.FileTransform
    from: "{{ inputs.file }}"
    script: |
      logger.info('row: {}', row)

      if row['name'] == 'richard':
        row = None
      else:
        row['email'] = row['name'] + '@kestra.io'

Create multiple rows from one row.

yaml
id: jython_file_transform
namespace: company.team

inputs:
  - id: file
    type: FILE

tasks:
  - id: file_transform
    type: io.kestra.plugin.scripts.jython.FileTransform
    from: "{{ inputs.file }}"
    script: |
      logger.info('row: {}', row)
      rows = [{"action": "insert"}, row]

Transform with file from JSON string.

yaml
id: jython_file_transform
namespace: company.team

inputs:
  - id: json
    type: JSON
    defaults: {"name": "john"}

tasks:
  - id: file_transform
    type: io.kestra.plugin.scripts.jython.FileTransform
    from: "{{ inputs.json }}"
    script: |
      logger.info('row: {}', row)

      if row['name'] == 'richard':
        row = None
      else:
        row['email'] = row['name'] + '@kestra.io'

Source file containing rows to transform.

Can be Kestra's internal storage URI, a map or a list.

Minimum >= 2

Number of concurrent parallel transformations to execute.

Take care that the order is not respected if you use parallelism.

A full script.

Format uri

URI of a temporary result file.

The file will be serialized as ion file.

​File​Transform

FileTransform