Need to create dynamic robots.txt file using the rewrite and API route handler, that is working fine in dev but returned cached data in Prod mode
caching should disable in production mode
**package.json
**
`{
"name": "frontend",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev --experimental-https",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@apollo/client": "^3.8.0-rc.2",
"@apollo/experimental-nextjs-app-support": "^0.4.1",
"@fancyapps/ui": "^5.0.22",
"@googlemaps/js-api-loader": "^1.16.2",
"axios": "^1.5.1",
"bootstrap": "^5.3.1",
"eslint": "8.47.0",
"eslint-config-next": "^13.5.4",
"formik": "^2.4.5",
"html-react-parser": "^4.2.0",
"next": "^13.5.3",
"react": "^18.2.0",
"react-bootstrap": "^2.8.0",
"react-bootstrap-icons": "^1.10.3",
"react-countup": "^6.4.2",
"react-dom": "^18.2.0",
"react-google-recaptcha": "^3.1.0",
"react-slick": "^0.29.0",
"slick-carousel": "^1.8.1",
"winston": "^3.10.0",
"yup": "^1.3.2"
},
"devDependencies": {
"sass": "^1.66.1"
}
}
code in route handler api/robots.js .. below code return json response of robots data
`import { robotsData } from "@/utilis/robotsData";
export const dynamic = "force-dynamic";
export async function GET(request) {
const robots = await robotsData();
return new Response(robots);
}`
config in next.config.js added source and destination for robots.txt file
` rewrites: async()=>{
return[
{
source:'/robots.txt',
destination:'/api/robots'
}
]
},
`
**Expected output ** out should be like below and gets updated if data changes in backoffice
`User-Agent: *
User-Agent: Googlebot
Disallow: /install/
Disallow: /umbraco/addf
Disallow: /views/fdgdfg
Disallow: /wwwroot/
Disallow: /uSync/
Disallow: /Smidge/
Disallow: /App_Data/
Disallow: /App_Data New data/
Sitemap: /sitemap.xml/`