= file
:type: input
:status: stable
:categories: ["Local"]



////
     THIS FILE IS AUTOGENERATED!

     To make changes, edit the corresponding source file under:

     https://github.com/redpanda-data/connect/tree/main/internal/impl/<provider>.

     And:

     https://github.com/redpanda-data/connect/tree/main/cmd/tools/docs_gen/templates/plugin.adoc.tmpl
////

// © 2024 Redpanda Data Inc.


component_type_dropdown::[]


Consumes data from files on disk, emitting messages according to a chosen codec.


[tabs]
======
Common::
+
--

```yml
# Common config fields, showing default values
input:
  label: ""
  file:
    paths: [] # No default (required)
    scanner:
      lines: {}
    auto_replay_nacks: true
```

--
Advanced::
+
--

```yml
# All config fields, showing default values
input:
  label: ""
  file:
    paths: [] # No default (required)
    scanner:
      lines: {}
    delete_on_finish: false
    auto_replay_nacks: true
```

--
======

== Metadata

This input adds the following metadata fields to each message:

```text
- path
- mod_time_unix
- mod_time (RFC3339)
```

You can access these metadata fields using
xref:configuration:interpolation.adoc#bloblang-queries[function interpolation].

== Fields

=== `paths`

A list of paths to consume sequentially. Glob patterns are supported, including super globs (double star).


*Type*: `array`


=== `scanner`

The xref:components:scanners/about.adoc[scanner] by which the stream of bytes consumed will be broken out into individual messages. Scanners are useful for processing large sources of data without holding the entirety of it within memory. For example, the `csv` scanner allows you to process individual CSV rows without loading the entire CSV file in memory at once.


*Type*: `scanner`

*Default*: `{"lines":{}}`
Requires version 4.25.0 or newer

=== `delete_on_finish`

Whether to delete input files from the disk once they are fully consumed.


*Type*: `bool`

*Default*: `false`

=== `auto_replay_nacks`

Whether messages that are rejected (nacked) at the output level should be automatically replayed indefinitely, eventually resulting in back pressure if the cause of the rejections is persistent. If set to `false` these messages will instead be deleted. Disabling auto replays can greatly improve memory efficiency of high throughput streams as the original shape of the data can be discarded immediately upon consumption and mutation.


*Type*: `bool`

*Default*: `true`

== Examples

[tabs]
======
Read a Bunch of CSVs::
+
--

If we wished to consume a directory of CSV files as structured documents we can use a glob pattern and the `csv` scanner:

```yaml
input:
  file:
    paths: [ ./data/*.csv ]
    scanner:
      csv: {}
```

--
======