Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Command Line Parameters
:description: Command-line parameters the Stackable operator for Apache Airflow accepts, with their default values and usage examples.

This operator accepts the following command line parameters:

Expand Down
1 change: 1 addition & 0 deletions docs/modules/airflow/pages/reference/crds.adoc
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
= CRD Reference
:description: CRD reference for the Stackable operator for Apache Airflow on the Stackable Hub, covering every field, default and API version.

Find all CRD reference for the Stackable Operator for Apache Airflow at: {crd-ref}{crd-ref-release}&operator=airflow-operator.
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Environment variables
:description: Environment variables the Stackable operator for Apache Airflow accepts, with their default values and what each one controls.

This operator accepts the following environment variables:

Expand Down
1 change: 1 addition & 0 deletions docs/modules/airflow/pages/reference/index.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Reference
:description: Reference for the Stackable operator for Apache Airflow, covering CRD fields, command-line parameters and environment variables.

Consult the reference documentation section to find exhaustive information on:

Expand Down
1 change: 1 addition & 0 deletions docs/modules/airflow/pages/troubleshooting/index.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Troubleshooting
:description: Common Apache Airflow problems on Kubernetes, covering task logs in Azure or S3, API workers, DAG syntax checks and LDAP logins after a major upgrade.

== Azure Blob Storage Logging

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
= Cluster Operation
:description: Cluster operations for Apache Airflow, covering how to pause reconciliation of a cluster or stop it entirely through its custom resource.

Airflow installations can be configured with different cluster operations like pausing reconciliation or stopping the cluster. See xref:concepts:operations/cluster_operations.adoc[cluster operations] for more details.
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Graceful shutdown
:description: Graceful shutdown of Apache Airflow schedulers, webservers and Celery executors, covering the configurable shutdown timeout per role.

You can configure the graceful shutdown as described in xref:concepts:operations/graceful_shutdown.adoc[].

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Operations
:description: Operating Apache Airflow with the Stackable operator, covering graceful shutdown, Pod disruptions, Pod placement and cluster operations.

This section of the documentation is intended for the operations teams that maintain a Stackable Data Platform installation.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Allowed Pod disruptions
:description: PodDisruptionBudgets the Stackable operator writes for Apache Airflow schedulers, webservers and executors, and how to disable them.

You can configure the permitted Pod disruptions for Airflow nodes as described in xref:concepts:operations/pod_disruptions.adoc[].

Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Pod placement
:description: Pod placement for Apache Airflow on Kubernetes, covering the default affinities and how Pods started by the Kubernetes executor are placed.

You can configure the Pod placement of the Airflow pods as described in xref:concepts:operations/pod_placement.adoc[].

Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Resource Requests
:description: Find out about minimal HA Airflow requirements for CPU and memory, with defaults for schedulers, Celery executors, webservers, dagProcessors and triggerers using Kubernetes resource limits.

include::home:concepts:stackable_resource_requests.adoc[]
include::home:concepts:partial$stackable_resource_requests.adoc[]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems broken:

Unresolved include directive in modules/airflow/pages/usage-guide/storage-resources.adoc - include::home:concepts:partial$stackable_resource_requests.adoc[]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Damn! Why did it work for me? Thanks. I'm on it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this against the latest docs main branch with my other PR? stackabletech/documentation#976

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, that works

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's good but now I'm suspicious. Which command did you use to try it?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make build-truly-local but the local documentation repo was not updated.
It worked after I updated the main branch.


A minimal HA setup consisting of 2 schedulers, 2 workers, 2 webservers, 2 dag-processors and 1 triggerer has the following https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/[resource requirements]:

Expand Down
Loading